Day 2 of #30 days of Testing
Starting reading Experiences of Automation
testing by Dorothy Graham and Mark Fewster. To preview the book please click Here.
Day 1 # 30 Days of Testing
What is Automation and what is Test Automation?
Automation:
Automation is
the technology by which a process or procedure is performed without human
assistance. Automation is the use of various control
systems for operating equipment such as machinery, processes in
factories, boilers and heat treating ovens, switching on telephone networks,
steering and stabilization of ships, aircraft and other applications and
vehicles with minimal or reduced human intervention. Some processes have been
completely automated.
And what is Test Automation?
In software testing, test automation is the use of special software (separate from the software being tested) to control
the execution of tests and the comparison of actual outcomes with predicted
outcomes. Test automation can automate some repetitive but necessary
tasks in a formalized testing process already in place, or perform additional
testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.
The
objective of automated testing is to simplify as much of the testing effort as
possible with a minimum set of scripts. If unit testing consumes a large percentage of a quality assurance (QA) team's resources, for example, then this process might be a
good candidate for automation. Automated testing tools are capable of executing
tests, reporting outcomes and comparing results with earlier test runs. Tests
carried out with these tools can be run repeatedly, at any time of day.
What is Continuous Integration and Continuous Deployment And ideal work flow
Continuous Integration
Continuous
integration is the practice of constantly merging development work with a Master
branch so that you can test changes and test that those changes work with other
changes. Idea here is test our code as often as possible so we can catch the
issues/bugs as early as possible.
In
the continuous integration process, most of the work is done by an automated
test technique.
Continuous Deployment
Continuous deployment eliminates the human safeguards
against unproven code in live software. It should only be implemented when the
development and IT teams adhere to production-ready development practices and
thorough testing, and when they apply sophisticated, real-time monitoring in
production to discover any issues with new releases.
Ideal work flow
Step 1: Developer checks in code to development branch.
Step 2: Continuous integration server picks up the change,
merges it with Master, and performs unit tests and votes on the merge to
staging environment based on test results.
Step 3: If Step 2 is successful, developer deploys it to the staging environment
and QA tests the environment.
Step 4: If Step 3 passed, you vote to move to production and the continuous
integration server picks this up again and determines if it’s ok to merge into
production.
Step 5: If Step 4 is successful, it will deploy to production environment.
Test an Application without Requirements
Method #1:
Work with whatever little documentation you can
get your hands on. It could be a basic simple backlog (in agile projects), an
email, an older version of the Business Requirements document, or old test
cases, etc.
Investigate, ask around and there is always some
documented trial.
When this does not work out, do not discount your
experience as a software user. For example, your real-life experience dealing
with online shopping experience and banking experience really counts here. I believe
you don’t need someone to tell you how to do online shopping or funds
transferring from one account to another account.
This really helps when you are testing some
shopping or banking application. I do agree that not all situations are going
to be straight forward however some time real life experience does helps.
Method #2:
Use the
older/current version of the application as a reference to test the future
release of a software product. Now, I admit this is in negation to the rule,
“Never write test cases using the application as a reference”. However, when we
are working in a less than perfect situation, we have to mold the rules to fit
our needs.
It helps
to keep the following aspects in perspective when doing so:
- The application might contain bugs- so if after registration the system directly takes you to Screen1 (a certain hypothetical screen for the sake of our example) – Never assume that is the correct behavior. Also if a field takes alphanumeric chars and is a phone number- a question that and make sure you do not take the application as a granted example for expected functionality.
- In the above situations use your judgment and take the help of the application to give you a jump start but be critical of it to question it’s working.
Method #3:
Talk to
the project team members:
- Offer to attend their meetings.
- Ask if you can participate in the unit and integration testing stages.
- If not, ask if the dev team can share their unit and integration test results.
- Arrange for a time for knowledge transfer at a convenient time.
When is Automated Testing a Good Idea?
• Regression Testing:
For re-testing pre-existing application functions that are being carried forward
to new versions (usually the majority, unless app is brand new)
• Smoke Testing:
For getting a quick high-level assessment on the quality of a build and making
go / no-go decision on deeper testing
• Static &
Repetitive Tests: For automating testing tasks that are repetitive and
relatively unchanging from one test cycle to the next
• Data Driven
Testing: For testing application functions where the same functions needs
to be validated with lots of different inputs & large data sets (i.e.
login, search)
• Load &
Performance Testing: No viable manual alternative exists
Importance of Manual Testing
- .Manually Testing Tests from Human Perspective
Human Testers can quickly identify when something looks “Odd”.
Automated test scripts don’t pick the visual issues. When tester interacts with
the software as a end user would, they able to discover usability issues and
other interface glitches. Automated test scripts can’t test these things
2 2. Exploratory Testing Can only Done Manually
Automated test only perform the actions that you tell them
to. To write Test scripts require planning and preparation to write, which
restricts the test to certain boundaries. These boundaries mean there isn’t any
room to stray from the written test to truly explore the application.
Exploratory testing
gives us the opportunity to answer questions like, “What happens if I do this?”
It enables to crave our own path throughout the test with little to no boundaries.
3 3. Errors/ bugs with Automated Test Scripts
Automated test scripts can contain bugs just like code can
have bugs. This means Automated Testing has the potential to report false
positives and false negatives. By adding human touch during the testing process
these errors can be avoidable.
4 4.Manual Testing helps us to understand the
whole problem/Application.
Without any doubt automated testing can save time and the
cost. Also it is great for getting quick results on broad level, however manual
testing allows us to understand the problem on conceptual and emotional level. Manual
Testing connects us with the end user and introduces us to level of empathy
automation doesn’t provide.
Sample Test Conditions and Test Cases along with the Automation Sample
I have created for sample Test conditions and Sample Test Cases along with the Automation on my Learning Process.
Links as bellow:
My Automation on sample project made available in Github. To access please click here
Subscribe to:
Posts (Atom)