- Making PyPI’s test suite 81% faster
- People aren’t talking enough about how most of OpenAI’s tech stack runs on Python
- PyCon Talks on YouTube
- Optimizing Python Import Performance
- Extras
- Joke
About the show
Sponsored by Digital Ocean: pythonbytes.fm/digitalocean-gen-ai Use code DO4BYTES and get $200 in free credit
Connect with the hosts
- Michael: @mkennedy@fosstodon.org / @mkennedy.codes (bsky)
- Brian: @brianokken@fosstodon.org / @brianokken.bsky.social
- Show: @pythonbytes@fosstodon.org / @pythonbytes.fm (bsky)
Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too.
Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it.
Brian #1: Making PyPI’s test suite 81% faster
- Alexis Challande
- The PyPI backend is a project called Warehouse
- It’s tested with pytest, and it’s a large project, thousands of tests.
- Steps for speedup
- Parallelizing test execution with pytest-xdist
- 67% time reduction
- --numprocesses=auto allows for using all cores
- DB isolation - cool example of how to config postgress to give each test worker it’s on db
- They used pytest-sugar to help with visualization, as xdist defaults to quite terse output
- Use Python 3.12’s sys.monitoring to speed up coverage instrumentation
- 53% time reduction
- Nice example of using COVERAGE_CORE=sysmon
- Optimize test discovery
- Always use testpaths
- Sped up collection time. 66% reduction (collection was 10% of time)
- Not a huge savings, but it’s 1 line of config
- Eliminate unnecessary imports
- Use python -X importtime
- Examine dependencies not used in testing.
- Their example: ddtrace
- A tool they use in production, but it also has a couple pytest plugins included
- Those plugins caused ddtrace to get imported
- Using -p:no ddtrace turns off the plugin bits
- Parallelizing test execution with pytest-xdist
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Ridiculous History
History is beautiful, brutal and, often, ridiculous. Join Ben Bowlin and Noel Brown as they dive into some of the weirdest stories from across the span of human civilization in Ridiculous History, a podcast by iHeartRadio.