Open Source Project Infrastructure is a Differentiator

Mon 22 January 2018 | tags: Open Source, Infrastructure, -- (permalink)

Open source software has evolved in multiple dimensions in the 15+ years I've been contributing to projects. One of the things that's significantly different in 2018 vs. 2003 is project infrastructure. For Hobu's open source software ecosystem, we've focused on providing industrial and professional tools for contributors coming to our projects. This infrastructure is a quality signal to potential users, contributors, and competitors. Like a stack of Yelp reviews, it gives people the ability to discriminate the project in relation to other competitors.

In 2003, project infrastructure was expensive and custom. Version control was the center of that infrastructure, just as it is today, but the next ring of support such as continuous integration testing, bug ticket workflows, mailing lists, and documentation systems were often a hodgepodge. OSGeo organized to help provide infrastructure to projects because it was so challenging for individual projects at the time. In 2018, a software project without that second ring is demonstrably lesser quality than its other open source peers that have it. Commercial software, of course, hasn't ever given you such visibility into its process.

PDAL Infrastructure

PDAL's project infrastructure starts with GitHub, as do most these days. The continuous integration platforms are both Travis for Linux and AppVeyor for Windows builds. Each commit triggers a full build and a full run of PDAL's integration tests. The documentation system and website is built on the fantastic Sphinx documentation system that's used for other project such as Python and MapServer. The PDAL Mailing List is generously hosted by OSGeo, and a Slack-like chat is provided by Gitter. These are all quite standard items that most projects have equivalents.

One of the things that's much easier to achieve in 2018 is automation of once manual drudge tasks. These events trigger off of commits/pushes to the source repository, and they give the development team leverage to focus on development. Every commit to the PDAL maintenance branch triggers the following automated activities:

  • The tests are run with Travis and AppVeyor.
  • The pdal/pdal Docker containers are built.
  • A full PDAL OSGeo4W package is built and uploaded to S3, ready to be pushed into the packaging system at release time.
  • The pdal.io website is regenerated with Sphinx and it is pushed to GitHub Pages.
  • A PDF of the entire website is generated with Sphinx to serialize the documentation and pin it to a release version while making it easy to use in environments where browser-based reading isn't as convenient.
  • Notifications are sent to IRC, Slack, and the pdal-commits email list.

Conclusion

Users making choices about whether or not to invest the time to learn open source software have a few things they can use to discriminate their choice. Project infrastructure is one thing that gives people confidence that your open source project is worthy of the effort. Public infrastructure is also a discriminator between open source software and commercial software. As always, project documentation is an important component of a project's infrastructure, but other items such as automated packaging, regression testing, notification, and deployment greatly enhance the value of the software. Automation also allows the project to focus on writing code and documentation instead of drudgery.

comments powered by Disqus