From 2dac64b39df375fbd9c07934a2d747fb54a05b79 Mon Sep 17 00:00:00 2001 From: Jacob Herrington Date: Mon, 16 Dec 2019 14:16:16 -0600 Subject: [PATCH] Expand the testing docs (#5112) * Add context to Acceptance Testing guide * Add docs on Regression testing * Expand CI skip docs * Expand documentation on testing controllers [ci skip] --- docs/tests/acceptance-tests.md | 24 ++++++++++++++------ docs/tests/readme.md | 11 ++++----- docs/tests/regression-tests.md | 35 +++++++++++++++++++++++++++++ docs/tests/skip-ci.md | 16 ++++++++++--- docs/tests/unit-functional-tests.md | 17 ++++++++++++-- 5 files changed, 86 insertions(+), 17 deletions(-) create mode 100644 docs/tests/regression-tests.md diff --git a/docs/tests/acceptance-tests.md b/docs/tests/acceptance-tests.md index 4d39b1cf2..fa3927423 100644 --- a/docs/tests/acceptance-tests.md +++ b/docs/tests/acceptance-tests.md @@ -6,11 +6,21 @@ title: Acceptance Tests Acceptance tests are tests from the perspective of the end-user. -It means that we are simulating what a user could do from their web browser and -testing the expected behavior of the app. +In the Rails world, we sometimes refer to these as Feature or System tests. In +Rails 5.1, a tool called Capybara was included to help us simulate a human's +actions inside of our tests. -Acceptance tests can be found in the directory `spec/system` (in Rails -terminology these are called "system tests"). +Generally, we are simulating what a user could do from their web browser and +ensuring that the app behaves as intended. + +When a feature is heavily reliant on interaction from a user via the browser, +it's a good idea to write automated Acceptance tests to uncover any bugs that +might not be apparent from manual testing. It's important to note that Rails +System tests can be fairly slow, so it's better to focus on testing core +functionality or pieces of your code that you think might be prone to +regressions. + +Acceptance tests can be found in the directory `spec/system`. You can run all acceptance tests with: @@ -32,6 +42,6 @@ bundle exec rspec spec/system/user_views_a_reading_list_spec.rb:10 where `10` is the line number of the test case that you want to execute. -You can read the official guide [Testing Rails -Applications](https://guides.rubyonrails.org/testing.html#system-testing) to -learn more. +You can read the official guide +[Testing Rails Applications](https://guides.rubyonrails.org/testing.html#system-testing) +to learn more. diff --git a/docs/tests/readme.md b/docs/tests/readme.md index 190682e7f..219e17352 100644 --- a/docs/tests/readme.md +++ b/docs/tests/readme.md @@ -6,6 +6,7 @@ items: - unit-functional-tests.md - preact-tests.md - accessibility-tests.md + - regression-tests.md - code-coverage.md - codeclimate.md - skip-ci.md @@ -31,13 +32,13 @@ a bug. Ideally, we test the functionality of the frontend and the backed. If you'd like to help us improve our test coverage, we recommend checking out our total coverage and writing tests for selected files based on SimpleCov's -test coverage results. You can also check out [Code Climate -summary](https://codeclimate.com/github/thepracticaldev/dev.to) which includes -the test coverage. +test coverage results. You can also check out +[Code Climate summary](https://codeclimate.com/github/thepracticaldev/dev.to) +which includes the test coverage. If you're new to writing tests in general or with Rails, we recommend reading -about [testing with Rails, RSpec, and Capybara -first](https://guides.rubyonrails.org/testing.html). +about +[testing with Rails, RSpec, and Capybara first](https://guides.rubyonrails.org/testing.html). ## Continuous Integration & Continuous Deployment diff --git a/docs/tests/regression-tests.md b/docs/tests/regression-tests.md new file mode 100644 index 000000000..8b6f55d19 --- /dev/null +++ b/docs/tests/regression-tests.md @@ -0,0 +1,35 @@ +--- +title: Regression Testing +--- + +# Regression Testing + +Regression testing is a technique for detecting breaking changes over time in a +codebase. + +Proper Regression testing is one of the most powerful tools in a programmer's +arsenal for avoiding the bugs that crop up as a project grows in size. + +Generally speaking, Regression testing is accomplished by a combination of +[Acceptance tests][acceptance_tests] and [Integration tests][integration_tests] +performed on areas where bugs have been found in the past. As it happens, +software bugs tend to exist in groups; writing tests around buggy code will +almost always uncover new bugs as code evolves. + +The practice of Regression testing usually takes place after code has been +written, and is used to detect if recent changes have introduced new bugs by +breaking previously functional software. + +There is no perfect prescribed method for Regression testing, and it has evolved +with the introduction of [Continuous Integration][ci] practices like the ones +that DEV uses. However, CI doesn't replace the need for Regression testing. +Regression tests should still be added to the codebase when bugs are discovered. + +If you submit a bug patch to the DEV application, you might be asked to write a +Regression test around your patch to help warn future DEV contributors if that +bug ever makes another appearance. This practice has the added benefit of +helping to ensure your patch fixes the bug. + +[acceptance_tests]: /tests/acceptance-tests/ +[integration_tests]: /tests/integration-tests/ +[ci]: /deployment/ diff --git a/docs/tests/skip-ci.md b/docs/tests/skip-ci.md index 5ab575d5d..a667c48e3 100644 --- a/docs/tests/skip-ci.md +++ b/docs/tests/skip-ci.md @@ -2,7 +2,17 @@ title: Skipping CI for minor changes --- -# Skipping CI build (not recommended) +# Skipping Continuous Integration -If your changes are **minor** (ie. updating the README or fixing a typo), you -can skip CI by adding `[ci skip]` to your commit message. +It's almost always a good idea to let our [Continuous Integration][ci] tools do +their work, but sometimes it can makes sense to skip CI. + +In the case of extremely minor changes, like updating the project README or +fixing a typo in the docs, you might want to skip CI by including `[ci skip]` in +your commit message: + +```shell +git commit -m "Fixed a typo in the testing docs [ci skip]" +``` + +[ci]: /deployment/ diff --git a/docs/tests/unit-functional-tests.md b/docs/tests/unit-functional-tests.md index 3c1fe2bde..9048b024b 100644 --- a/docs/tests/unit-functional-tests.md +++ b/docs/tests/unit-functional-tests.md @@ -36,5 +36,18 @@ bundle exec rspec spec/models/user_spec.rb:10 where `10` is the line number of the test case that you want to execute. -You can read the official guide [Testing Rails -Applications](https://guides.rubyonrails.org/testing.html) to learn more. +## Testing Controllers + +Historically, it has been common to use Rspec to write tests for Rails +controllers. This pattern isn't necessarily discouraged in the DEV codebase, but +Rspec has introduced a more effective way to test controllers via Request Specs. + +Request specs test the actions on a controller across the entire stack, +effectively acting as Integration Tests. You can read more about request specs +in our documentation on [Integration Tests][integration_tests]. + +You can read the official guide [Testing Rails Applications][rails_guides] to +learn more. + +[integration_tests]: /tests/integration-tests/ +[rails_guides]: https://guides.rubyonrails.org/testing.html