In a software development project, it is a natural process for several developers and testers to work on different modules. Each developer or tester will be assigned a different module, so it is extremely important to test and build the modules to ensure they work seamlessly with each other with zero bug interference.
How do you test the user interface for detecting discrepancies and bugs?
This is done through a process known as Continuous Integration (CI). Many software development companies already use CI with great success.
To put it simply, CI is a methodology that caters to the building and testing of development work to identify and address the pain points before they become serious issues.
In the past, the standard procedure was a single daily build, but in the current working ambience and requirements, this is done frequently, with only a few hours between tests. This gives a consistent feedback on the project’s status, so problems are solved before they become complex.
So each time a product code is developed, it goes through an automatic CI process. There are several tools for Continuous Integration to help trigger these builds and tests. We will go through those later.
Why is CI Testing Important?
The main aim of Continuous Integration is to avoid integration challenges. Integration, in software development, is the practice of integrating individually created software components to create a whole product. These components are integrated into subsystems and the subsystems themselves become one whole product.
Why you need CI is quite simple to understand. A software development project necessitates the relationship between developers and management. One of the core values of this relationship is trust. The management gives the requirement of a particular application and the developers agree to deliver the completed version of the same on a particular day.
They reach an agreement with the management taking the developer’s promise at face value failing to realize that problems or challenges might hinder with the development progress. In the end, if there is a failure, the management feels that the developers have betrayed their trust, thereby straining their relationship, with the outcome reflected adversely on the product.
This situation can be completely avoided through CI process. Each team member has an idea of what the other person has done so far, so there will be no contradictions or confronting situations.
Each time a developer makes a commit to the repository, automated test runs will be conducted. This helps in identifying the breaks or bugs in the code, even identifying the person who caused the breaks.
Continuous Integration helps in identifying the smallest breaks in the application code and fixing them early in the development stage. Hence, each integration is verified.
Keeping this in mind, let’s have a run through the various advantages of CI:
1. Early Detection of Bugs
Continuous Integration helps in locating bugs early in the development process. Since you can detect at an early stage, you can solve problems quickly and spend quality time building features.
Failing to follow a continuous integration process may prove to be costly because you may have to spend long hours trying to find the bugs. With CI, you no longer have to worry about long and tense integrations.
The CI server is responsible for monitoring the repository for changes in a build or bug occurrence. But it is the responsibility of the team to monitor for changes.
2. Helps in Determining Code-break
Through CI, code breaking is detected. The developer makes his commit to the build, and automated testing process takes place to ensure the new code doesn’t break any of the existing codes. He will also wait for his peers to make new check-ins.
If a conflict arises, then he will have to fix it so the new code and the changes that came before and after are synced. This way, you can increase code quality.
3. Only a small number of changes are lost when the code base reverts to a bug free state
We already know that CI helps in detecting bugs and as the tests are conducted automatically, with the developer making frequent checks, it is easy for the code base to get back to old, bug-free state. Only very few/minimal changes will be lost as the integration is taking place frequently.
This is a time and money-saving feature, hence a major advantage. When a unit test fails, the team can go back to the original bug-free state as well, without debugging.
4. Regular Availability of a “Current” Build
Another advantage of CI is the continuous and regular availability of an updated build (give or take a few hours, probably, between the last test) to help with testing, demo activities and even release purposes.
When you need to test or release your software, it will automatically get the current version of your software, rebuild it and then run it. The developer will receive alerts or notifications regarding the success or failure of the build-and-test.
5. Enables Continuous Deployment
When the result of the development process moves towards deployment through Continuous Integration, it leads to a production-like environment. The automated functional testing executed at regular intervals leads to continuous deployment followed by continuous delivery.
This leads to a good build, which in turn, pushes the product into production. The whole process is automated, with no human intervention, and continuous deployment happens several times a day.
Example: Gitlab.com goes through multiple continuous deployment process. Each time a commit is made, the code would go through test and build process, merged into the main branch and deployed to the product environment.
6. Enables Automated Testing
As mentioned in the previous point, the testing is automated, while the changes are deployed automatically into the production pipeline. Each time an integration is made, it would be verified by an automated build, thereby reducing integration problems and helping the team to develop error and bug free software rapidly.
There are automated testing tools for this. Automated testing leads to continuous testing, thereby saving hours of developer time (when done manually), speeding up development life cycle and aiding in continuous delivery process.
7. Reduces the Risk of a Longer, Time-Consuming Project
Using CI process for continuous delivery and automated testing is definitely speeding things up. What used to take weeks can now be finished within hours.
When you are working on a huge, risky project, integration problems can bite off a huge chunk of your schedule. So, it’s really important to do CI testing.
8. Increases the Quality of Your Software
Through CI, you are eliminating the blind spots completely, helping you release a bug-free and error-free software. The software meeting client requirements, is delivered on time, minus the usual hiccups with performance and with no compromise in quality.
The risk of missed deadlines is no longer a cause for concern because there are practically no coding defects. Of course, the fact that clients’ demands remain unchanging also helps a lot.
Best Practices of CI Testing
Renowned British software developer, Martin Fowler has laid down 11 key fundamentals for an effective CI Testing.
1. Maintain a Single Source Repository
It is important to put in everything for a build in a single source repository, so the source code management system gets easier. Everybody in the team will know where to go if they want a particular file.
2. Automate the Build
Automate the tasks in software development to avoid mistakes and to save time. Include everything in the automated build and this means getting the database schema from the repository and running it in the execution environment.
Using build tools can help analyze the changes made while conducting the required tests.
3. Everyone Commits to the Mainline Everyday
Integration is all about communicating at the right time. Committing frequently and regularly would help the developers to break their work into easy, workable chunks, spanning a few hours.
4. Build Every Commit to the Mainline
When every commit is built to the mainline, it remains healthy, up-to-date. Whenever a commit is made, the server checks the sources in the integration machine automatically, initiates the build and notifies (of the success or failure) the person who made the commit.
The notification will mostly be sent via email. The aim is to find problems as and when they arise, so continuous monitoring is important.
5. Keep the Build Fast
As mentioned earlier, rapid feedback is the key. The commit build has to be good for the project to move on, so it is important to keep the build fast. You can probably start at a one-hour build and then move closer.
6. Make the Build Self-Testing
Self-testing of the build help catch the bugs and is particularly useful in the XP and TDD environment. You need a suite of automated tests to self-test the code, and the test should give out the result.
7. Fix Broken Builds Immediately
Through continuous build, you can be alerted on the mainline build failure and take responsive action to fix it. It is normal for the mainline build to break, but make sure it doesn’t happen all the time.
8. Test in a Clone of the Production Environment
Create a mimicked production environment to test the code. Make sure you use the same operating system version, same database software with same versions to create the test environment. Use test doubles to ensure the test pace is normal.
9. Make it Easy for Anyone to Get the Latest Deliverables
It should be easy for anyone involved in the project to explore the changes made in the code, the status of the project and so on. There must be a common location for people to find the latest deliverable.
10. Everyone can See What’s Happening
Nothing beats visibility. Yes, it also depends on the tools you are using, everyone involved in the projects must be able to see the state of the system, the changes made in the code and the main build state. You can put in indicators to check this status of the build.
11. Automate Deployment
You need to develop scripts that would move the tests between different environments automatically. As it is you have multiple environments in CI, one for running commands, another for running secondary tests and so on. So, doing this through automated processes would reduce the tension.
In addition to the above practices, make sure you do integration testing before unit testing and keep the testing suites for both separately.
Common Tools for CI Testing
There are several tools for Continuous Integration, and most of them are open source.
It is open source and ideal for custom continuous build process. Comes with an extensible framework.
CI tool from Atlassian. Build, tests and deploys tests automatically. Unleash the potential of agile development, detects bugs.
Building and testing software projects continuously and monitoring the run jobs. We will get into Jenkins in detail later.
Contains a master and repository of slaves. The master monitors the codes and assigns tasks to the slaves. Was intended to be an alternative to Tinderbox.
Open source, free and very easy to use. Offers SaaS version to help with free testing.
Written in NodeJS and JS, uses MongoDB as a backing store. Customization can be done through plugins.
Created and open sourced by ThoughtWorks. Distribute your builds across various systems and later monitor them.
Jenkins – the Best Tool for Continuous Integration
Written in Java, Jenkins is the most popular Continuous Integration server in the developer world today. The fact that it is open source, flexible and extendable, makes it easy for the developer to initiate the builds.
Additionally, Jenkins has an ecosystem of more than 1100 plugins enabling you to use support projects in almost any language of choice, over large databases and in almost any version control systems. Jenkins is free, supported by a huge user community and runs in Apache Tomcat, a servlet container.
Jenkins has evolved to be in perfect sync with Continuous Integration and Continuous Delivery (CD) pipeline. We know what CI and CD can do but with Jenkins, it becomes entirely automated, right up until deployment.
To use Jenkins, you need to have an accessible source code repository containing your codes and a good build.
CD ensures that you can use all the merged codes for software production with Jenkins. So, it actually acts as a middleman between your the code repository and the build server. As it checks the server for code changed every 3 minutes, it is completely reliable.
Extensibility – One of the best things about Jenkins is that it can be extended to contain any number of plugins.
Rich Plugin Ecosystem – Users can add all sorts of functionality, thanks to the rich ecosystem of plugins. Third-party plugins can be configured in Jenkins to incorporate additional functionality.
Manages the Development Side of DevOps – Jenkins can be used for everything, right from source code management to Continuous Delivery. DevOps by the way, is a term used for depicting two major related trends.
It combines Agile operations, practices and tools with software developers and other IT professionals collaborating with each other to help in the automated software delivery process.
Integrates well – Jenkins helps you to integrate with a growing number of testing and deployment technologies for Continuous Delivery. The integrated ecosystem of Jenkins helps in coordinating the software infrastructure aiding in Agile development processes.
Jenkins allows the developers and other IT teams to focus on their part of the job, while it keeps automating the build, increasing productivity. Errors and bugs will be instantly notified so if there are remote developers working on a project, monitoring becomes easier. File fingerprinting allows to keep track of dependencies, jars and various build versions as well.
Automated Testing Using Selenium
Selenium is a suite of different software tools that enables automated testing of web applications across various browsers/platforms. Originally developed by Jason Higgins, it is open source and hence, widely used.
You can automate selenium testing with Jenkins, but first you need to install Selenium.
Normally, QA Engineers uses only those tools that would help with their project, but you need to know the different tools to approach different testing environments and automation problems.
Follow the steps below to write your first test case using Selenium WebDriver and executing it using TestNG.
- Right click on the src folder ->New -> Package.
- Provide package name something like com.stm.test and click “Finish”.
- Right click on the newly created package – > New -> Class.
- Provide class name as “RegistrationTest” and click Finish.
- Write the code for your first test.
To see an example of how this is done, visit http://www.softwaretestingmentor.com/writing-your-first-test-case-in-selenium-webdriver/
- After finishing the test, right click on the test and click on RunAs – >TestNG Test
- After executing the test, select the project and press F5 to refresh the project. A new folder “test-results” will get created which will show you the results for the execution. Right click on index.html->open with->web browser to see the execution report.
To automate tests, you can follow these steps:
- Identify the tasks
- Incorporate input data to be created
- Define the results of the application in any expected order so it works correspondingly
- Execute the test
- Compare results with expectations; this would decide whether the test was successful.
Integrating Selenium with Jenkins
It’s relatively easy to run Selenium tests with Jenkins build when you are using frameworks like TestNG and JUnit. Just follow these steps to integrate Selenium with Jenkins.
Part 1 : Download Jenkins
- To use Selenium with Jenkins, you need to download Jenkins first. Visit the official website https://jenkins-ci.org/ to do this. Configure Jenkins by following the steps by opening Command Prompt.
- Download Jenkins.war file. Save it where you retrieve it quickly. The desktop would be a good choice. If your download is complete, then you will need to take Jenkins.war file.
- Next, go to the directory where jenkins.war is installed. Enter the command – java –jar Jenkins.war.
- Start- cmd>Project_home_Directory> java -jar Jenkins.war. This would get Jenkins to run (provided you did the configuration correctly)
Part 2 : Configure Jenkins for Selenium
- Retrieve Jenkins from where you have saved it.
- Open Command Prompt (MCD) and start Jenkins. If everything is fine, you will get the message that Jenkins is fully up and running.
- Jenkins, by default runs on 8080 port, so open any browser, type URL http://localhost:8080 to get Jenkins UI running. This step would get Jenkins working.
- Create build that would help Jenkins running. You can also configure Jenkins to recognize other tools like Java and Maven as well.
- Click on “Manage Jenkins”.
- In the Manage Jenkins window, Click on “Configure System”. To integrate Selenium with Jenkins, you need to tell Jenkins where the Java is located. This is why this step is important.
- Navigate to JDK section and Click on “Add JDK” button.
- Uncheck “Install automatically” option so Jenkins will only take Java which we have mentioned above.
- You can give names like JAVA_HOME and Specify the JDK path. Incorporate the email feature for timely notifications. This is optional, but you can configure it and click on Apply.
Your Jenkins is now configured.
Part 3 : Execute Selenium using Jenkins
Test cases in Jenkins can be executed in four different ways:
- Execute Windows batch command
- Execute shell
- Invoke Ant
- Invoke top-level Maven targets
Here, the example will follow Window batch command:
Step 1 – Create a Batch file and Add it to Jenkins
- Set class path of TestNG to execute testing.xml files.
- Open command prompt and set the classpath
Home directory > set classpath=C:\Users\Learn-automation\bin;C:\Users\Learn-automation\libs\*;
Depending on system, make changes as you go.
- Open notepad and type the below command and save as .bat file – for example, run.bat
java -cpbin;libs/* org.testng.TestNG testng.xml
Step 2 – Now Create a Job in Jenkins that would Execute the Build
Open Jenkins on browser (type http://localhost:8080)
- Click on the new item.
- Give a Job-Name, select Build a free-style software project option and Click on OK button.
- Navigate to Advanced Project Options > Check the use custom workspace > in directory specify the project home directory.
- Specify the Add Build step >Click on Execute Windows batch command.
- In the section, please specify the batch file which we created and click on Apply and Save.
Step 3: Run the Build and Click on “Build” option.
Step 4- Check Build History and Console Output and Verify the Output
Part 4: Schedule the Build in Jenkins for Regular Execution
Schedule the build for existing jobs by following the steps given below.
Open job which we created now and Click on configure > select the check box build periodically.
Specify the time here we need to be careful about the syntax.
Jenkins will accept 5 parameters to test your build. Let’s look at each of them
- first parameter– specify minute and range will vary from 0-59
- second parameter– specify hours and range will vary from 0-11
- third parameter– specify day and range will vary from 0-7 here 0 and 7 will be Sunday
- fourth parameter– specify month and range will vary from 1-12
- fifth parameter– specify year so here you can specify *
1.If you specify 00 22 * * * it means your build will run daily @ 10 PM
2.If you specify 50 * * * * it means your build will run daily 50 min after every hour
3.If you specify 00 22 1 * * it means your build will run every Monday @ 10 PM
If you follow all the above mentioned steps, you can run Jenkins to test your build successfully.
Credits: Learn Automation
Software development processes are gravitating towards Agile practices, emphasizing the importance of Continuous Integration. The margin of success enjoyed by your company will be significantly higher followed by pleasurable working environment and finally, satisfied customers.
And then, there is the added advantage of Agile development. If you want to make changes in the business logic, you have the flexibility to do it anytime.
Continuous Integration helps you to release your product on time, with irregular and late software releases becoming a thing of the past.
Make sure that your app meets the best quality standards with Continuous Integration testing. Our expert test engineers are there to help you.