Monorepo readings and thoughts semi-consolidated

27-02-2025

Goals:

• reduce duplication (ie scaffolding, boilerplate, configuration)
• simplified dependency management and packaging
• increase developer productivity
• reduce context shifting / cognitive load

Blog posts:

• Opendoor
  • https://medium.com/opendoor-labs/our-python-monorepo-d34028f2b6fa
    • package structure, editable installs, CI/CD, builds + releases
  • https://gist.github.com/dan-hipschman/070318806610727f17b2d6616ceaa4cc
    • generates TOML file, converting local dependencies to versioned dependencies
• Tweag
  • https://www.tweag.io/blog/2023-04-04-python-monorepo-1/
  • https://www.tweag.io/blog/2023-07-13-python-monorepo-2/
• Pinterest
  • https://medium.com/pinterest-engineering/building-a-python-monorepo-for-fast-reliable-development-be763781f67
• Airbnb
  • https://www.youtube.com/watch?v=sakGeE4xVZs
• Uber
  • https://www.youtube.com/watch?v=lV8-1S28ycM
    • monorepo
      • good; faster IDE, mercurial, submit queue, buck instead of gradle
      • bad; IDE lag, git slowdown, broken master, long build times
    • multirepo
      • good; strong ownership, fast build times, separate git repos
      • bad; silos, functionality duplication, dependency hell
• Twitter
  • https://github.com/twitter-archive/commons/tree/master

Examples:

• https://github.com/carderne/postmodern-mono
• https://github.com/dermidgen/python-monorepo/tree/master
• https://github.com/charbonnierg/repo/tree/next

Tools:

• python
  • https://github.com/carderne/una
  • https://pypi.org/project/poetry-multiproject-plugin/
• build tools
  • https://bazel.build/
  • https://www.pantsbuild.org/
• distribution
  • https://github.com/indygreg/PyOxidizer
  • https://github.com/pex-tool/pex
  • https://github.com/pyinstaller/pyinstaller
  • https://github.com/marcelotduarte/cx_Freeze
• Github
  • https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_iddefaultsrun
  • https://medium.com/@SammyRulez/git-monorepos-with-github-actions-e47f56d8793b
  • https://brunoscheufler.com/blog/2022-04-24-required-github-actions-jobs-in-a-monorepo

Misc:

• https://twitter.com/jasoncwarner/status/1592227285024636928
  • past CTO of GitHub / Heroku
  • monolith > apps > services > microservices
  • think of code as debt, and services as an extreme form of debt