Austin: Hi, everybody. Welcome to today’s webinar, Make the Shift from Manual to Automation with Open Source. My name is Austin. I’m from Perfecto Marketing and we have a really exciting webinar today. We’re joined by Uzi Eilon, Perfecto’s Director of Technology, as well as Siva Anna, Senior Director of Enterprise QA at Apexon. Welcome everyone.
Before I pass it over to them, just a few quick housekeeping items; we have a community panel and we’d be happy to answer your questions during the webinar or after if we don’t get to the questions. So please submit if you have any questions. It’s best to watch the webinar in full view. So you can do that in your WebEx panel. After the webinar, we’ll email you the slides from the recording, so watch out for that email in the next day or two. Then once the webinar is over, we have a survey. We’d love for you to answer those questions to give us some feedback on how to improve the next webinar. Okay, without further ado, I’ll toss it over to Uzi.
Uzi Eilon: Hi, everyone. Hi, Siva. How are you?
Siva Anna: Good morning, Uzi. Good. I am doing very good.
Uzi Eilon: Good morning everyone. We are really excited to have this webinar. So today we’re going to talk to you about the challenges of mobile automation, the challenges of mobile projects, the challenge of how to create real automation unattended. Then what we want to do is talk about framework. What are the requirements for a framework? After that, we are going to introduce you QMetry, which is a very strong framework that Perfecto by Perforce selected to use and, on top of that, we created a new project that we are going to share with you.
Then I would like to talk little bit about BDD and explain why BDD is very important for automation today. How does it connected between the different people in the organization through the automation flow. Then we’re also going to show a demo. We believe that every webinar needs to contains a real demo. So we’ll show you everything and at the end, if we’ll have time we’ll answer a few questions.
So let’s start and let’s start with the challenges. So the first part of the challenges when we are talking about automation and automation script is first of all how to create automation? We found that it’s very hard to define or to identity the right objects. I analyzed hundreds and thousands of test cases and I found a lot of issue with the implementation of the objects. This problem like not robust enough, object ID or something like this, can affect the robustness of the automation and can affect the maintenance. How much time do you need to spend in order to improve your script to support more devices, to support other execution?
Another thing that we find that it’s really hard to automate is the complex actions in the application space. So right now, the application becomes more complex. The action with the mobile becomes very sophisticated, so sometime we need not only to run the automation on our application, but to use an external API and to use an external feature of the device. So this actually makes the automation very hard. Then, last but not least, special events; call, alerts, pop-ups etc., through the automation always affect your automation.
Remember what we want to achieve is unattended automation. We would like to make sure that we can have automated script that can be executed unattended through the CI. We talked a lot about the creation, but I think what is harder is to maintain the script. Why? Because through diversion and, if we are talking about mobile automation, we are always talking about agile project, it change all the time. You have new releases. You have new object ideas. Now some devices behave differently based on their actions or based on the special feature on the device, different platform, different browser, and rapid change in the application.
As I mentioned, there is a feature that released. It worked all the brand got input from the app store and today all the brands are actually monitoring the app store. They are changing it rapidly, so all of these facts make the maintenance very hard. So if you don’t have a strong automation, built in a very structured framework, it will be very hard to maintain. So that’s one challenge that we would like to face today. The second one is about the mobile. Yes, we are creating a mobile automation and mobile scripts and mobile automation, everything on the mobile application.
So we talked about the new feature of the mobile or the advanced features of the mobile, but we’re also going to talk about the different version, the different screen size. If we are talking about responsive web, a lot of the screen size, the specs, dramatically on the way that your web looks on the device. If you’re talking about the native application and usage of a specific API feature from the device, the version affect dramatically on your application. We also talked about the less time to develop.
Last, but not least, all the new devices contains a new options like BLE, Camera, GPS. All of these features makes it really complicated to develop, but even harder to test, because there are different GPS driver on each device. Siva, I know that you are working with a lot of projects. What I would like to ask you here, based on the projects that you work with, what is the percent of your projects actually use feature around the advanced automation feature like GPS or camera?
Siva Anna: Well, Uzi, it’s good that you asked this question. I mean nowadays, every mobile application that we develop or test in the market, we pretty much see these features how they become as a de facto standard. Pretty much every application have some of the features as part of the application.
So some features include mostly the camera-based function, QR code scanner or bar code scanner or we see the mobile wallet, touch ID, so all these features, right? So day-by-day there are new use cases coming and for automation this has become so that any work out solution that you select, that needs to support these advanced features. I mean these are not all advanced. It’s just becoming a basic feature.
Uzi Eilon: That’s very important to mention here that all of these feature need to be automated as well. So if you have a test that need to open the camera and read a bar code, how did you automated it? We added some feature in Perfecto by Perforce tools which implemented in this framework which allow you to test it. It’s called Image Injection. You can go to PerfectoMobile.com to read more about this specific feature and if you have and want to more about it, let us know through the questions and we will return with you, return to you with the answer. But right now, we would like to focus on the high level, how to create the test, and not on a specific feature.
The third challenge is the different persona in the organization. So today the organization are working together. So we have the developer that write the code. We have the tester most of the time and what we saw lately that the tester and the developer are part of the same team. So suddenly we have the same teams that will work together and we have also someone which is the product owner, which defines the flow. Each of the persona can actually add some data into the test flow and into the test requirements. How do you do this?
How do you take persona which is the product owner that understand the flow, but he’s not technical, and have the same language with the tester, which know how to test but does not know anything about writing in Java, with the developer that knows how to create all this writing and all the magic in code, but he doesn’t understand the customer needs? How do you create one option or one language to connect them all?
So this is a challenge that we thought about it a lot and I think we find a very nice solution for that, based on BDD. We were going to talk about it later. The third challenge, which is very important, is actually the execution. So let’s say I have automation and I have my execution. How do I execute this to the furthest? How do I execute it unattended? What happen if the device is in use? Who is going to handle it? How does it works? I have 100 test cases and ten devices.
How do I manage this execution? More important from that, let’s say that I have hundreds of test cases. I’m going to execute all of them and I’ll get 93 failure. I’m going to analyze. How I will be able to find exact error based on the report? So we found that the ability to execute the script unattended, multi-threaded, and to get the results and analyze it, it’s a very important feature in the framework. I mentioned CI and Siva, here I want to talk to you a little bit about the CI. What are the differences that you see between mobile CI and standard CI?
Siva Anna:Uzi, I think, I mean if you had asked me this question maybe a couple of years before, there used to be a difference between the typical web application versus mobile applications, but with the advancement in the Cloud offerings like Perfecto what it offers to the customers, we are seeing no difference between the web applications and the mobile applications.
So basically you are allowed to, not only the continuous integration, but you’re also how mechanism to do the continuous deployment to the devices. We can build the binaries for both iOS and android and other platforms and you can take it to the device, install it on the fly, and you can then do a smoke test or the whole integration test. So with the advanced offering that are available in the market today, the continuous integration and the continuous deployment is no different between web application and mobile application.
So it’s available to take it all the way to some other test app stores also that completely validate the application and deploy it to the app store so it can be allowed for any customers to test it out. That is pretty advanced and the CI in the mobile application is, in a way it is unique that it brings the device a variation in terms of the OAS and the compact. Definitely there is advance and it is very much possible.
Uzi Eilon:Thank you for that. I want to highlight actually two points that you just talked about. First of all, the magnitude of the execution, so if we have one test and we have ten devices, so suddenly we need to execute it ten times. So let’s think those projects that they have hundreds of test cases and tens of devices. So we have a lot of execution. So that’s one point. The other point which I found very, very important is actually that it’s the same as the standard CI process.
So we cannot come and change the internal process in the organization and this framework is part of the requirement was to use the standard tools that today exist in the organization in order to expand the CI process and not replace and change all the processes. So based on everything that we just said, we defined in Perfecto by Perforce a few requirements for a tool, few requirements from framework, in order to offer it to our customer.
Before we define it, what we wanted to make sure that this framework is an open source. Why? Because we believe that the framework is something that need to be open source, need to be shared, need to beget some input from everyone in the community. So we talked about open source. We talked about open and connected. Perfecto by Perforce allows you a lot of way to connect to devices, a lot of way to connect to a specific and unique command.
We would like that this framework which allows us to connect to the specific tools, but sometime you also want to run it on your local device and not only on Perfecto by Perforce. So we think that and that’s part of the requirement that these tools will be able to run on devices in the Cloud and also on local devices. Another thing that we found very important, we are talking about mobile devices. We are talking about execution of tests. We need to execute it on all of the available devices, so multi-threaded was very important for that.
For the persona, remember we designed all the different personas, so we looked for an open source which allows us to use BDD because we believe that BDD have to improve the conversation between the different personas in the organization and allow to build a full SDLC with the different personas in the organization. Another thing that we found very important is we wanted to base on the standard tools first of all because we don’t want to invent the wheel.
We believe that if someone creates a good framework or a good part of the framework, we should use it. Second, as we mentioned, we need to connect to existing processes. So if we are talking about CI server, a lot of people use Jenkins. We need to use Jenkins. We are not going to develop a new CI server. We looked for a tool that allows us to create easy script and to be very structured in order to have easy maintenance. We looked for a strong report capability and, as I mentioned before, we wanted to be able to run on the SaaS and on the local.
So we found QAF and, in the next slide, Siva is going to describe a little bit what QAF and what we can do with QAF. On top of QAF, we created a project named Quantum. Quantum is implementation of automation project, based on QAF, which allows you to connect better with Perfecto by Perforce, which allows you to create an easy script, and which allows you to manage your executions and multi-threaded executions. Both are open source. They are linked to both of the projects and, in the end of the webinar, we’ll show you what will be the right next steps in order to start working with these tools. So Siva, please tell us little bit about the QAF.
Siva Anna:Thanks, Uvi. So I think we saw what some of the challenges that Uvi described and also that’s what we see, right? So the QMetry Automation Framework, in short QAF, that what’s in design we continue to enhance the features. It was primarily built for a single objective, single most objective, of having a new kind of automation framework that allows you to automate the different channels.
So nowadays we see applications span across multiple channels. So the automation framework, the QMetry Automation Framework, was primarily built around to support so that you don’t have to go through learnings of different techniques, based on each channel that you want to automate, whether it is a desktop web application or mobile applications, whether be it mobile web or native or hybrid. You would have to go through a learning or having a different approach.
So that was the single most objective that we had when we started doing and as we see more and more requirements and the needs in the market, we continue to enhance, to provide the experience that we initially thought about it. So as you can see, this is the open source GPL based license. You can see the link and we’ll continue to give more information about this framework as the Perfecto implemented framework on top of the QR called Quantum. So we continue to give this information all along the demo, as well as in the follow up emails.
So to talk about what is the QMetry Automation Framework, right? It’s a ten year tough work. We started in the early days of Selenium. We saw several gaps in terms of how the object repository is being maintained, what kind of reporting that is being offered as part of the Selenium. So we saw several gaps in the early days of Selenium, and we started building framework on top of Selenium, right? I think you saw in the slide that Uzi mentioned that community automation framework is a collection of a few open source solutions, like Selenium, TestNG, and Appium.
All those are the components in which the community’s built upon. But we bring a lot of customization, right? So how do you author your test cases? Whether you want to author the test cases in BDD or keyword base or you want to build a Java based approach, the community automation framework, that support the different flavors of test authoring. So BDD, which much more option, especially in agile basedproject, where you have to start doing your automation or start doing the instant automation.
So BDD has become a very handy tool so you can write your test cases, but also it brings a lot of other benefits, like Uzi talked about, for different personas. So BDD is a lot more readable. So it can be read and understood by the non-technical users as well. We saw there was a need for having a BDD. This is also, like I said, this is a ten years. So we did some of these concepts, we have started even before cucumber was out there in the market.
One of the other things that we saw or we continue to improve is how you can reduce the maintenance because that was one of the key challenge that Uzi mentioned in the first slide. So maintenance, in our experience, we see two-thirds of effort goes into the maintenance. So that’s a key constant battle that we have, how we can minimize the maintenance. The maintenance comes in the form of optimized objects repository and how you can increase the reusability of the test code.
So some of those basic features that you would expect out of your framework, that’s what we’ve implemented. We continue to provide integration to various solutions that are out there in the market. Like for example, Selenium and Appium are the two popular tools, but we would be looking at it and see how you can take this one standard framework across the different solutions that might be there. So we’re looking to see how you can extend this framework somehow to some IoT testing or some other non-UI based testing as well. Uzi, can you move to the next slide, please? Thank you.
So this is very high level features. Like I said, we started with the focus on providing Selenium-based UA automation but we’ve come a long way that we test mobile applications as well as the web servicesThe web services allows you to do a lot of validations as well as you can start certain transactions or the use cases from mobile or the desktop channel and you can do some verifications by making the service call. So we have seen all these kind of complex scenarios. We have seen in the market also that customers tend to use different tools.
This was the one reason that we built a tool so that you didn’t have to have different tools. You can visit the QMetry website, where you can see the detailed reporting and a lot of trending analysis also available as part of the framework that allows you to see, pinpoint where the exact failure is. So especially if it’s a mobile device, you can see what was the screenshot? What was the device characteristic in which that you executed. All that information is captured out of the box, without you writing a single line of the code. So that’s a beauty and benefit of using your framework, like QMetry Automation Framework.
Uzi, can you please …? This is kind of the core features or the benefits of using Automation Framework. Like I said, I keep repeating this, but having a unified automation framework solves many of the standards or many of the problems associated with standards or approaches or how do you want to train the team, how you can collectively look at the execution data in one place. So that’s a core benefit you get by using a unified framework like QAF.
So the resuable test asset data is another key area that we have seen because of the beauty and also having reusable test data improve the code faster creation, as well as (audio cut out) minimize the maintenance associated with case automation effort itself. The data-driven is another key area because these are very, very common things you will see in any given framework. So data-driven is another key thing because we have seen even cucumber lack some of the data-driven testing support because it doesn’t allow you to use some external data sources.
That’s what we have addressed like four, five years before so that you can have your BDD and still you can continue to use external data sources, whether it is CSV, JSON file or even, for that matter, database itself. These are some of the benefits and continuing on the last slide before Uzi can talking about the demo and architecture of the framework itself. We believe in open source and we believe in open technology, right? So the QAF does allow you to provide various integration from automation tools to the CI tools versus test management tools.
So pretty much any tools or technologies that you might want to connect, it is available. It is pretty open technology, open architecture that you can integrate with any solution that you want to use. So having said that, Uzi you can continue. You can take control and you can give more details about the architecture and the demo itself.
Uzi Eilon:Thank you, Siva. Actually, based on everything that Siva just say, I just want to tell you a little bit about the process that we did in Perfecto by Perforce. So we analyzed a lot of frameworks. I just want to return to the previous slide. You can see that there is a lot of benefit in QMetry. So together with all the requirements that we defined, which have been answered by QMetry, all the easy way to use it. The ability to have the data-driven, the ability to connect between the different part of the organization, we find it very suitable to Perfecto by Perforce.
What we did we built a project on top of this. So right now, I would like to drill down into the project and to talk a little bit about the part of the project. We split into three different area. We have the script creation. It’s actually very similar to the problems that we found. These are the solutions. So the script creation. First of all, we defined or we understand from our customers that they want to use their internal ID. They have the IntelliG, they have the Eclipse, some of them has Android Studio. They want to keep and work on this Android Studio. This is what was very important for us, the ability to work with any ID.
Then when you have this ID, when you open your ID, it’s located with about how to create the script. First of all, you need to define few object repository. So first of all you need to have an object spy, the ability to get all the objects from the devices, the ability to press on a button and get the map of the dome object or the application object in the device itself. After you have this object spy, you need to build your object repository. That will be the key for your automation. We are going to drill down to a specific example about the object repository but, if you are building a good object repository, it make the maintenance very easy to use.
Let’s talk about the code. So we have different users that can create a different code. Some of them want to use Java and they will be able to use Java because Java is very easy for developer and some tester to create an automation. All you need to do is to define your object driver and then work with it. But we also find that other users, more test their product owner wants to create some automation. So we embedded the BDD and we selected the standard Cucumber BDD.
So you can have a BDD test, a Cucumber test, and it’s going to be part of your test. Together with that, we make the BDD very easy to use by adding a specific command to Perfecto by Perforce. Why? Because BDD is something that you can edit commands to the language. You can make the language reachable. So what we did we just made a lot of commands that connect to Perfecto by Perforce. Press the button on a device, swipe on the device, open the camera, install the application, all of these very hard actions now provided to you as part of the automation and, together with that, the BDD allows you as a developer, you as a smart tester, to create a logical action.
So if you need to define the login, and the login is like I don’t know a few action like find the user, enter the user, enter the password, validate something, you can create it as a Java code and add it to the BDD. So all the BDD user will be able to use this existing code. So the BDD is the ability to use English to create automation, the ability to get pre-defined actions that connected to the device in order to make it easier for you, and the ability to add some logical actions to use. Of course, all of this built on Appium and Selenium. We do find that Appium and Selenium are the automation standard in the field itself. So everything is built on the Appium and Selenium.
Let’s talk little bit about the execution. So in the execution we have the power of execution done by QAF, which knows how to get all the features, connect it to the objects from the object spy, and to execute it on the test. But this is also built on common tools, like TestNG and Maven and we have of course Jenkins. All of these are part of the automation. All of these tools are part of this execution part and this is the way that we connect it to the CI. So if you are creating a script and let’s say that you are creating your one script and now you want to execute it on 20 devices, it’s very easy. Why? Because this is part of the framework.
If you need now to handle devices in a different state, devices that are new, devices that execute different tests, it’s already handled by the framework. All the execute part is actually handled by the framework. That’s something that I would like to show you today. It’s very easy to take one script that you created and suddenly, not only to execute it, but also to connect it to CI and to execute it every night and to get a result. If you are talking about a result, Perfecto provide a new reporting tool, which is a SaaS reporting tool, which upload all the results into a private area that you will be able to analyze and learn about your execution.
This report tool allows you to execute or to get data not only from Perfecto reports, but from any test that you want and if you just want to have an API test or you have a QI test, even a manual test, you will be able to add some growth into this report tools and this report tools allows you to see all the results and to drill down to a specific action. So this is what we have here in Quantum. So it’s an open source project, built on top of QAF, use all of these common tools and makes your automation environment very easy to use. All you need to do is to focus on your test; the execution, the results, the advance options handled by the framework.
We know how to run and execute automation. We know how to collect the data. We don’t know how to create your own test. We gave you very easy BDD tools to create your test. So let’s see what is BDD, how to use it, what does it mean? So this is how we believe the test should look like in English. So let’s go on this test. Let’s log into Perfecto by Perforce. So what you can see here that I have my scenario. The scenario is login and I have typed, which is part of my smoke test. So if we want to execute only one tag, it want to execute the test based on a tag.
So I can use this execution manager to run my smoke test, my migration test, my full test, whatever I want. Nothing need to be handled by me. Everything is handled by the framework. So you can see this test is part of my smoke test set. It’s named login. Then I have two actions. Enter. I connect into Perfecto by Perforce. I’m finding the object’s name, field user, and I’m entering my password. I’m finding the field password and then I enter the password. At the end, I’m pressing on the submit button. What are these objects? We’re going to talk about it in a second.
All of these actions are already implemented by Perfecto by Perforce. You don’t need to do anything in order to use this action; press key, find elements, scroll down, all of these, install application, close application, all of these commands are already implemented. All you need to do is just to use your ID in order to pick it. It’s very easy and I’m going to show you how easy it is to create a script. So this is the way that script create. What I want to talk now is about how to add a logical action. So as I mentioned, we implemented all the action.
For example, when I click a specific object, you can see this is actually what we did in Perfecto by Perforce. We have the driver in our test. We are finding the object, and then we are clicking on the object. So this is something that we already implemented. It’s part of the framework. But let’s talk little bit about your logical action. Let’s say, “I would like to accept the user agreement.” In order to do that, what I need to do I need to scroll all the way down and then click on the button.
What I give you or what the frame will give you is the ability to do it very easy. What you need to do is to said, “Okay, I have my user agreement function. This is a job function.” The driver will be accessible to you all the time. All you need to do is to use the getDriver and the commands are available to you in the util library. So the open source contains this util library which can help you to create the scripts and also there’s a getDriver command, which allows you to get the driver. Think about multi-threaded, think about hundreds of devices.
Every time that you are writing the code getDriver, you are going to get the right driver for your test and. At the end, I’m clicking on the button agreement. So my test look like this. This is my and my test look like I accept the user agreement as part of my test. Remember I told you about we talked little bit about the object repository. What you can see here is that the object repository is actually connect between logical name of my sphere and the right ID or the right identifier. For example, I’m opening the object spy. I learned that my submit button can be defined by this xpath I created it on an external file and then, in my script, I’m going to use the logical action.
Let’s say tomorrow that this field’s going to be changed. I will be able to change it in this external file and all my tests will continue to run. More than that, and this is also part of the framework, if you have the same test but let’s say that the submit button is different between iOS and android, you can create two repository files, which contains the same logical name, but the different object ID for iOS version and the framework will be able to get the right object from this external repository. So it helps you to reuse your test and create scripts once and it helps you to maintain the script.
All the action to get the logical action, to get the right identifier, to go to the driver, it finds everything done by this framework. A few things about the framework. So the framework, Quantum framework, is full structure. You have these slides in the links or in the getting start links that I’m going to share with you, but there are a few actions that I just want, or a few libraries or folders that I would like to share with you. You have the library to create your own test, your own scenario. You have the library to configure the execution. You have the library to configure the object repository, and that’s it.
These are the three libraries that you need to use. Define your own objects repository, define your test, and then define the execution. The repository contains one more file, which is the Cloud or your driver definition area. In this case, you can say that I define Perfecto by Perforce as my Appium or Selenium grid. By the way, if you are using with local device, you can just enter your local address. If you are using other grid, you can change it. You can see everything is very, very standard. We are using the standard automation. So if you want right now to execute your test and use local device, just enter here server 127.0.0.1 and continue to work with your local device.
Last, but not least, the execution manager you need to have part of the XML for each execution. In this case, you can see that I defined two devices. I want all my tests to be executed on these two devices, which I defined by the name. I want one of them will be iOS and the other one will be an android. So what we want to do now, I would like to take you to tour with Quantum in the next ten minutes and to show you how does it look in my environment. So I do have a project. In this case I selected intelligent. In my project, you can see I created a test that said, “I would like to test the Amtrak train. I’m going to Amtrak.com.”
By the way, the framework contains and supports web and mobile application both. In this case we are going to see a mobile application. So I would like to go to Amtrak.com. I would like to click on the train status to find a station, to enter value to a specific station, and to submit and then to press the status button. By the way, if I want to edit command, you can see that all the commands are defined here.
So if I want to have a validation, I can said, “Okay I should see this as green. If I want to open an application to verify something on the application, version on the application date, you can see that all of these commands already implemented by the framework. So the framework allows you to use this test. So you see how easy it is to add the test. All you need to do is just to use this create define function. So again, if you want to start a new test, all you need to do is to define this object. Now this flow can be created not by the developer, not by the tester, but also by the product owner.
So this is one way, the way to create the script. You have to remember BDD and the ability to reuse data that’s already created by us and you can get it from a pre-defined list, that one. The second thing is the object repository. So if I’m going into the Amtrak, you can see that I defined my object repository. For example, if I want to press on the status button, I can see that my status button, this submit button, that I identified by the ID. So this is the xpath. So I created my object repository. I created my test and the next thing that I need to do or the last thing that I need to do is to define the execution.
So now I have two tests, but I would like to execute it on four devices. So I’m going into my third area. So this is my test area. This is my object repository area and this is my third area, the execution area. I creating a very simple file. I’m going to say the following, “I would like to execute all the tests with the Amtrak and tags…”–Let’s return to the test. You can see that it’s going to run the first and the second script–“… and I would like to execute it on Galaxy device, on iPhone 6, and on two specific device.” So the way that I identify the devices is very flexible.
You can do it by name. You can do it by any other property and that’s it. Then from here, I’m just running a standard TestNG. So what happen right now, I just execute it and the test is about to run on all of the devices on the same time. So I have a script that’s been executed on four devices and you can see right now, we are going to Amtrak on all of the devices. We are going to press on the find train station. We are going to press on the location and this is my test. So I have one test that I created with BDD. I used my framework to execute it on four devices.
It took me, the execution after I created the script, it took me like five minutes to configure the environment, to configure the execution. Now it’s a Maven project. I can take it and connect it to any CI tools that I want. So I have my execution right now and I have my test that runs on four devices. Again, standard test, standard scripts runs on four devices from my execution environment. You can see that I get the reports directly into my ID. We talked a little bit about the report.
So we also have the report that’s created by the reporting environment and the reporting environment is the new SaaS reporting tools, which allows you to see all the tests we just executed. So you can see that today I executed my Amtrak test. I would like to get only this Amtrak end test, so I can understand how many tests have been executed. What if it doesn’t say that my test actually passed today. I had some issue. We are going to drill down into the issue in a second, but let me try to understand better what’s going on or what happened during my execution.
So I would like to group my test by device and I can see I have a lot of issue on all of the devices. Right now at the end, it started to work on the iOS. So if I want to drill down to a specific execution, all I need to press is on the execution itself. What I found, and actually it’s a real issue, when I execute this test in the morning, I found that after going to Amtrak.com, when I press on the next steps, which is the buying a ticket starting like this, the M mobile site was down. This is why I got the error on all of the devices. Now it’s been fixed and right now I will be able to see this script that I just executed.
Another thing that’s very important for me to show you that the script tags are the same as your test. So if your test contains the word, I open the browser, I click on the train status, I found the station. The report itself will contains the same sentence. I click on the buy ticket, I enter the value. So everything that’s been executed reflect in the report and you will be able to see immediately what happened on the device, when you execute on a specific test. Of course the contents web and video and everything that’s related to Perfecto by Perforce.
So I would like to summarize this demo. We are giving you an open source to create a very easy script to create and maintain an object repository and to be able to execute it easily on a lot of devices and connected to the CI and get very strong report mechanism. What I want to do now, I would like to return to the slide and go over a few questions that’s been sent in the Q&A questions. So do we have some interesting questions?
Austin:One question from our audience is, “How can I use only the reports part?”
Uzi Eilon:Oh, so yes you … so, as I mentioned, this framework contains a lot of parts. If you want to have one part, it’s doable. If you want to have only the report part, if you want to have only the execution part, only the BDD part, you will be able to use it if you are a Perfecto by Perforce user. There is an API to use the report. The API works in two levels; one is rest API and the second one is if you are using testNG, we have links to testNG which create everything automatically.
Austin:Another question from our audience, “Can we use QMetry for both desktop and automation? In other words the script for desktop can be used in mobile browsers?”
Uzi Eilon:That’s a perfect question and the answer is yes. As I told you before, when we analyzed the framework, we looked for a framework that support everything. So yes, you can run it. I don’t know if you’re familiar with this feature, but we have a browser in Perfecto by Perforce. So you can run it on your local browsers, you can run it on a desktop, you can run it on mobile devices, and you can execute it on web in the Cloud. So everything’s supported, by the way. If you have a different tool, you can use it with the different tools.
Austin:Another question from our audience. “Can I use it without BDD?”
Uzi Eilon:Oh, that’s also a good question. Yes, if you want to write a Java code, you will be able to write just a Java code. I don’t want to use the BDD. I got it from a lot of developer and tester that feel comfortable with Java and they still need and get a lot of benefit from Quantum. Why? Because they’re focused on the Java code. All they need to do is just to use the getDriver and start the test and the Quantum framework handled everything; the execution, the report, and everything. Yes, it’s been used also without BDD. Any more questions?
Austin:One more. “How can I start with this framework?”
Uzi Eilon:That’s a very good question because it’s leaded me to the next slide. So yes, there is a link. It’s all open, so there are some links that show you how to work with. So this is the link for Quantum.com. In order to start, we have something called Quantum Starter Kit. This project is a full project that can be implemented in Maven or in Eclipse or in IntelliG. All you need to do is go over this getting start and work with it. By the way, you will be able to see that. We do have a document, a full document how to start, and this is the step-by-step.
You will see a full explanation how to get a project from GitHub, how to upload it to your repository, what can be done, what configuration need to be done. It’s really, really easy way to create a first project. After you’re having the first project, you can start and add more after. If you want to use QMetry, you also have the options to download it from here. Again, also QMetry contains a lot of documents. That was very important for us when we are looked for a framework, which will be well-documented.
So you have all the documents that you need about QMetry and you have this getting start project in Quantum that allows you and teach you how to start and create an automation. By the way, I also challenging you. If you find it as an open source and if you find some commands that’s used by you … If you created a very cool command, you can add it to all of the users by committing to the project. So this project is open for you.
So I’ll give you an example. We just released last month the image injection feature, which allows you to take an image and send it to the camera so when you run your automation around the image, when you run your automation with the camera, we can send any image that we want. We didn’t add this command and one of our customer added a command and committed to the code. So if you find something interesting, if you want to add some code to Perfecto by Perforce, Quantum, you will be able to do it. If you want just to start and try it, go here.
Go into the document, download the step-by-step, follow the step-by-step details which shows you first how to download and install it and how to create your first BDD script and that’s it. You will be able to run some automation. Anymore questions?
Austin:A few more questions came in from our audience, one from Valerie. “Can this be used in Visual Studios and ID? You mentioned that this was built on open source, but then the reporting area is an SaaS?”
Uzi Eilon:So that’s also a very important solution. So let’s talk little bit about the report. So this is one part of the report QMetry contains a very strong report, local report, which is part of the open source. So if you want to use this report environment, it’s also available. So when I’m executing the test, I have the SaaS report, but I also have the QMetry report that you can use and these are local reports environment.
Austin:Okay, everybody. Well, thank you for attending our webinar. Again, we’ll send out the slides and recording in the next day or so and any questions were unanswered, we’ll also follow up with those very shortly.
Uzi Eilon:In the last slides, you will be able to get all the links for how to start QMetry and how to understand what’s going on there and how to start Perfecto Quantum.
Austin:Alright. Thanks again. Bye.
Siva Anna:Okay, thank you. Thanks for attending the session.
Uzi Eilon:Thank you, Siva.
Siva Anna:Thank you, Uzi.
Austin:Bye, bye.
