Testing Software at DrChrono

Testing Software @ DrChrono

In 2022, with Python 🐍 and beyond 🛸

🤖 How familiar are you with testing software?

I was born in it, molded by it

I've written one before

I know some of these words


Go to app.deckdeckgo.com/poll and use the code {0}
Awaiting votes

🧪 Test (noun)

  • A procedure intended to establish the quality, performance, or reliability of something, especially before it is taken into widespread use.

😪 The Problem

  • Add new features without breaking existing features.
  • Fix existing bugs without introducing more problems.
  • A reliable product that will prevent customers from churning.

💖 Advantages

  • Have confidence when you make changes.
  • Introduce less defects.
  • Catch defects sooner, when the cost of fixing them is the least.                                     


🛑 Limitations

  • Opportunity cost. Writing tests is an investment of resources that could be used for new features or fixing bugs.         
  • Other tools like mypy and code review can catch many of the same problems at a lower cost.
  • Tests can make changes harder.

🏭 Testing is quality control for our software factory

PyCharm's Test Runner tab

  CircleCI runs our tests every time we create a Pull Request, and all checks are required to pass

⚙️ An example

Using the iOS app, a provider tried to filter out appointments for their Sacramento office, but visits from other offices were showing too.

1️⃣ The actual fix was only a single line!

📼 Betamax: Record your test suite's HTTP interactions and replay them

🖼 Front-end tests

  • We don't have any tests for our JavaScript, Angular, or JQuery code.
  • Elm has some tests, but we're stuck on an older version and have decided to replace it.
  •  TypeScript and React have good test coverage.

🛤 TestRail

🛤 TestRail

🤖 Selenium

📐 Measuring the quality of quality control

  • “If you cannot measure it, you cannot improve it.” - Lord Kelvin
  • Test coverage is a measure (in percent) of the degree to which the source code of the program is executed by a test suite.
  • The first published reference was by Miller and Maloney in 1963.

💯 Should we try to achieve 100% test coverage?

💵 How much testing is economical?

  • https://hypothesis.works/articles/economics-of-software-correctness/
  • The problem is not that we do not know how to write correct software. The problem is that correct software is too expensive.                        
  • The amount of testing required is different for every business, it depends on the impact of failure.

🚀 The SpaceX Dragon's displays use JavaScript and has a "high bar" for test coverage

🤨 How do I achieve 100% test coverage?

An easy way is to write tests before you write the code.

📚 Recommended Reading! This is free online at obeythetestinggoat.com

🧬 Mutation Testing

  • Another way  to measure test quality is to add "mutations" to you code and run your test suite
  • A test suite that does not detect and reject the mutated code is considered defective.
  • Mutation testing gives insight into whether you adequately exercised the lines your tests cover and adequately asserted on failures

🔮 Opportunities for the future @ DrChrono

  1. ​Selenium code to automate Test Rail scenarios.
  2. Test the untested JavaScript, or prioritize it to be rewritten in Typescript.
  3. Test and lint our Terraform and Salt code.
  4. Experiment with mutation testing

👋 That's all folks!


GDC. (2021). Automated Testing of Gameplay Features in 'Sea of Thieves'. YouTube. Retrieved December 29, 2021, from https://www.youtube.com/watch?v=X673tOi8pU8.

Percival, H. (2017). Test-driven development with Python: Obey the testing goat. O'Reilly.

Prause C.R., Werner J., Hornig K., Bosecker S., Kuhrmann M. (2017) Is 100% Test Coverage a Reasonable Requirement? Lessons Learned from a Space Software Project. In: Felderer M., Méndez Fernández D., Turhan B., Kalinowski M., Sarro F., Winkler D. (eds) Product-Focused Software Process Improvement. PROFES 2017. Lecture Notes in Computer Science, vol 10611. Springer, Cham. https://doi.org/10.1007/978-3-319-69926-4_25

MacIver, D. R. (2016, April 15). The Economics of Software Correctness. Hypothesis. Retrieved December 29, 2021, from https://hypothesis.works/articles/economics-of-software-correctness/

R/spacex - we are the spacex software team, ask us anything! reddit. (n.d.). Retrieved December 29, 2021, from https://www.reddit.com/r/spacex/comments/gxb7j1/

Quality control employee at a gun factory with thought balloon that reads,? - New Yorker Cartoon. Tom Cheney Published May 16, 2005