Auto-Convert Freestyle Jenkins Jobs to Coded DevOps Pipelines

Learn how to migrate from Jenkins 1.0 to 2.0 seamlessly

Sanil: Thank You Sarah. Good morning everyone? Today we are going to be looking at an important plugin in the Jenkins community. The goal of the plugin in helping migrate from 1.0 to 2.0. In terms of the agenda, we’ll just talk about what is the motivation for the plugin and why Jenkins 2.0, what does the plugin actually do. A quick demo to show you some of the features of the plugin. Then, we will deep dive into what is the scope of the plugin? What are the areas where you might need customization et cetera. Finally, we’ll wrap it up with some value-added services for the plug-in that Infostretch provides. Let me quickly get started.

Let’s start with Jenkins 2.0. I’m sure a lot of you on the call already have started the migration paths from 1.x to 2.0. But if not there are very good reasons for you to consider 2.0. To start with pipeline as code, the first- class citizen within Jenkins 2.0. What that means is a lot of the freestyle jobs that you had earlier in Jenkins 1.x now can actually be written as code. Once you have the entire pipeline job written as code you get a lot of the benefits of having version control, having a good elegant architecture for the pipeline, multiple teams working on it and every software engineering principle benefits that you can derive, you can actually get in pipeline as code.

The other important part Jenkins .0 is actually the stage view. Stage view again is first-class citizen. You get a better view for how the pipeline flows, how various jobs are, where the bottlenecks are et cetera.

Finally, your freestyle job is to continue to function in 2.0. Migration does not necessarily mean that all the jobs are going to be broken but definitely there is a big motivation to actually move to a coded pipeline. That is where essentially the plug-in comes into play.

If you look at the typical problem and let’s say and be working with some customers who actually have about 100 to thousand jobs in freestyle that they actually are converting in the coded pipeline, when we looked at the problem statement and when we talk our customer it’s very clear that the process of doing that today is actually a bit of brute force. Where you have a lot of people put on the project. They’re looking at each of the freestyle jobs that actually writing coded pipelines for each of the jobs.

As you can see from the top flow here, that process definitely has got some problems and to begin with, it is really manual and so it’s error-prone. It is, the other problem essentially that takes a lot more of time. If you can do 10 jobs and convert them into coded pipeline and then think of scaling that to about thousand jobs, you can immediately see that the amount of time you’re going to take to actually convert them, get it tested, it’s going to be much larger compared to a tool that can help you do it in a much optimal way.

Finally, you need to really understand a lot of the nuances of freestyle job to actually convert them into coded pipeline. What that means, you need a lot more skills within your team who could actually be spending time building other parts of your DevOps ecosystem, now you take those resources and actually have them spend time building the pipeline.

Given these set of problems in a manual conversion, what we decided was to put together an open-source plug-in that can actually auto-convert to your freestyle jobs for you. Essentially, the benefits from the plug-in, they directly relate to some of the downsides of doing it manually. To begin with, it is definitely automated. There is no human intervention to a large extent. It is relatively bug-free. There is a significant effort reduction.

In terms of ROI, you could be looking at definitely saving about 70% of time that you would actually spend doing it manually for most of the jobs that you have. It’s highly customizable. A couple of slides down the line, I talked about why that is important, because different customers have different DevOps pipelines freestyle job setup and there are certain need for customization.

Having talked about the plug-in let me give you a quick overview of the plug-in, how it works. It is available in the marketplace. Obviously, you can download it and you can test it out. But let me show you how the plugin works and quick demo with that. What I’m going to do here is, so this environment that you’re looking at right now actually does have the plug-in installed. Once you have the plugin installed, you have a couple of options. One is that you can go and take a particular freestyle job and actually convert them or you can actually go and convert an entire folder. You will end up getting an option here saying convert your folder jobs to Pipeline.

I’m going to use this option today. I’m going to click on that. What you get here is that you have a way to name the job. I’m just going to call it Jenkins Plugin Webinar demo. Then, you can actually choose the job that you want to convert. Now, you can see that there are multiple jobs here which are feature jobs that are waiting to be converted. You can start from any point of time. Now, in this particular case these jobs are actually chained together. What that means is that a build job will be followed by a static analysis job, will be followed by an exit job.

There is a chaining of the job, feature job within. You can have any starting point for your conversion. In this case I’m going to use a WebApp-Build. You have an option of saying there, do I want to recursively go and convert all the downstream jobs or do I want to convert one job.

In this particular case, I’m going to convert all the downstream jobs too, which is most of the times what you want to do because you want the plug-in to actually take that burden from you and let it actually convert all the jobs in a way that links it in the final Pipeline too. Right. I’m going to choose this option and then convert it. Now you have another option here where you can actually go and check in the file into your get deposit radar.

You could either get the entire snippet of code created in line or you could actually have a Jenkins code that gets actually checked back into your get deposit radar. That means you post option where you can see the Jenkins file. Let’s do the conversion. You can see the conversion runs really fast. There is no job running here right now. I’m going to just build this job while I show you what the converted job looks like.

If I go to configure for the particular converted job, this job actually is a converted job from the Freestyle one. A lot of the options that you see here have been transposed over from the Freestyle job. If I go all the way down, you can see that a snippet of code got created. This is essentially a groovy script code that got created for you. All of the entries here, for example the node that was chosen, the different stages that you have here, they all got converted over from the Freestyle job.

As an example, you can see that there is a stage called the WebApp-Build which if you recall is the one that I started with for the conversion. If I go further down, you can see that another stage has been created for a unit test and so on and so forth.

The conversion tool is going to automatically link this app for you it’s going to look at a lot of the options that you have in your freestyle job and then convert it into code and ready for you to be used via pipeline.

If I go back again to my job quickly. This is the one that we were running. You can see that it’s already started the process it’s going to continue through all the different stages and complete it for you. This is a quick demo of the tool and how it works when you download it from the Jenkins Plug-in community you’ll see there a lot many other options that you can actually use within the plug-in.

This is a quick snapshot of the Open Source Jenkins Plug-in Community page what it looks like. You can see that there have been quite a few installs — and we released it pretty recently in the marketplace. We have your test running it internally with some of our projects and now we have released it available for you to use within either your Open Source Jenkins Community or even your CloudBees community or CloudBees installation.

A quick slide on what are some of the features that would see available in the plug-ins. As you saw you can either convert a folder of jobs to the pipeline or could also convert a single freestyle job to a pipeline. In most of the cases you would end up doing the former because you have a lot of jobs that needs to be converted. As I mentioned both with Open Source Jenkins and also the CloudBees Enterprise Jenkins, you can further customize it as required.

This is an important aspect because when we work with clients we do often see that the freestyle jobs have a standard– standard tool — the toolchains that are used but they are also very specific new answers they have created for their job. The plug-in has been designed to take care of the lot of common cases for conversion and obviously it cannot cater to all individual customer scenario, given that is the case, we can customize the plug-in further to meet individual client needs. If it is something that is useful for the community at large then always that will get rolled into the Open Source Plug-in Community itself.

The other important aspect that when you’re using CloudBees and using that old based active control the plug-in does adhere to that and make sure any security constraint that you had within your freestyle job is being moved over to the coded pipeline and any access that you had earlier and are not broken, it takes care of that very well. This is an important aspect of what is the scope of the plug-in? As I mentioned earlier, it tries to do a lot of the things automatically for you but it takes your freestyle, notes setup take a freestyle parameter, take most of your standard job and moves them over.

Then, it also allows you to connect to get right now as a repository, it will take some of the environment probably that exist of which in most common properties and then of course, a lot of the bills step in the possible actions. The list of possible actions that production that you see here are all converted for you automatically. We do understand that you may have very specific possible actions very custom to your requirements but those can be done– those will not be done by the plug-in but in cases where those exist, we will put placeholders in your job so that you can actually go in and modify them manually and customize them manually.

After having installed the plug-in, if you’ve used it for hundreds of your jobs even thousand jobs conversion, you will find that it does a good job of converting a lot of your jobs to coded pipeline but often there are cases where you need help either on customizing the plugin for very specific scenarios or you may need help just with training on how to use the plug-in and how to best deploy it within your ecosystem.

Finally, an important aspect of conversion whether it’s manual, whether to the tool is to really test out the converted pipeline and make sure that it is doing what it’s supposed to do. Each of these can be standardized so they can be very custom requirement, Infostretch just provides services to help with all the three.

If you have a scenario where you need either customization or testing or training Infostretch provide very specific services and expertise to help you with that.

Sarah: Great. Thank you so much, Sanil. At this time we would like to get your questions answered and as a reminder you can at any time after the question in the Q&A window at the center of your screen, looks like we have the first question here. Sanil what are the advantages of a pipeline as code?

Sanil: That’s a very good question, the reason why pipeline as code is very important is because once you put in your entire logic for your job as code, you’ve got a lot more flexibility in deciding what the flows should be, what aspects of it should be changed, you’ve got modularity and as I mentioned earlier, a lot of the good software engineering principles that you get out of writing good elegant code you get out of the system, you will get multiple people working on the pipeline. It could actually be– you could have it checked into your source control system and have multiple people work on it together. But these are definitely a great benefit of doing pipeline as code. The flexibility and also having a great elegant architecture.

Sarah: Great Thank you. You have another question here for you. What support is Infostretch committed to provide for the plug-in?

Sanil: Sure. Infostretch as an important practice around DevOps and as part of practice for DevOps, we are very determined to make sure that any tools which need our support and especially the plug-in that we developed is striving and is constantly announced, but for the help for the community. We do have constant support both on the bug fixes and enhancement side and also on any kind of help training side for this plug-in.

Sarah: Perfect. Thank you. And another question here. What percent of jobs can be converted using the plug-in?

Sanil: So if a lot of the jobs are very standard in nature using largely standard tools in the tool chain, typical post-built actions et cetera, most of it can be converted. But in a very typical scenario, what is seen is that you have at least 70% of the jobs that can be converted and there could be another 20% which needs some minor tweaking to be done, and maybe 10% which would need some major customization. So that is a good split to look at when you think about how many can really be converted.

Sarah: Great thanks to Sanil; and I believe that’s all the questions we were able to answer at this time and I just want to thank everyone for joining us today. You will receive an email in the next 24 hours with a slide presentation and link to the webcast replay and if you’ve any questions, please contact us at info@infostretch.com or call 1-408-427-1100 to speak with the representative.

Lastly, we encourage you to download our free plug-in, we listed it right there in the chat box for you. So please, go ahead and do so. Thanks again everyone. Enjoy the rest of your day.

Learn how to migrate from Jenkins 1.0 to 2.0 seamlessly

Sanil: Thank You Sarah. Good morning everyone? Today we are going to be looking at an important plugin in the Jenkins community. The goal of the plugin in helping migrate from 1.0 to 2.0. In terms of the agenda, we’ll just talk about what is the motivation for the plugin and why Jenkins 2.0, what does the plugin actually do. A quick demo to show you some of the features of the plugin. Then, we will deep dive into what is the scope of the plugin? What are the areas where you might need customization et cetera. Finally, we’ll wrap it up with some value-added services for the plug-in that Infostretch provides. Let me quickly get started.

Let’s start with Jenkins 2.0. I’m sure a lot of you on the call already have started the migration paths from 1.x to 2.0. But if not there are very good reasons for you to consider 2.0. To start with pipeline as code, the first- class citizen within Jenkins 2.0. What that means is a lot of the freestyle jobs that you had earlier in Jenkins 1.x now can actually be written as code. Once you have the entire pipeline job written as code you get a lot of the benefits of having version control, having a good elegant architecture for the pipeline, multiple teams working on it and every software engineering principle benefits that you can derive, you can actually get in pipeline as code.

The other important part Jenkins .0 is actually the stage view. Stage view again is first-class citizen. You get a better view for how the pipeline flows, how various jobs are, where the bottlenecks are et cetera.

Finally, your freestyle job is to continue to function in 2.0. Migration does not necessarily mean that all the jobs are going to be broken but definitely there is a big motivation to actually move to a coded pipeline. That is where essentially the plug-in comes into play.

If you look at the typical problem and let’s say and be working with some customers who actually have about 100 to thousand jobs in freestyle that they actually are converting in the coded pipeline, when we looked at the problem statement and when we talk our customer it’s very clear that the process of doing that today is actually a bit of brute force. Where you have a lot of people put on the project. They’re looking at each of the freestyle jobs that actually writing coded pipelines for each of the jobs.

As you can see from the top flow here, that process definitely has got some problems and to begin with, it is really manual and so it’s error-prone. It is, the other problem essentially that takes a lot more of time. If you can do 10 jobs and convert them into coded pipeline and then think of scaling that to about thousand jobs, you can immediately see that the amount of time you’re going to take to actually convert them, get it tested, it’s going to be much larger compared to a tool that can help you do it in a much optimal way.

Finally, you need to really understand a lot of the nuances of freestyle job to actually convert them into coded pipeline. What that means, you need a lot more skills within your team who could actually be spending time building other parts of your DevOps ecosystem, now you take those resources and actually have them spend time building the pipeline.

Given these set of problems in a manual conversion, what we decided was to put together an open-source plug-in that can actually auto-convert to your freestyle jobs for you. Essentially, the benefits from the plug-in, they directly relate to some of the downsides of doing it manually. To begin with, it is definitely automated. There is no human intervention to a large extent. It is relatively bug-free. There is a significant effort reduction.

In terms of ROI, you could be looking at definitely saving about 70% of time that you would actually spend doing it manually for most of the jobs that you have. It’s highly customizable. A couple of slides down the line, I talked about why that is important, because different customers have different DevOps pipelines freestyle job setup and there are certain need for customization.

Having talked about the plug-in let me give you a quick overview of the plug-in, how it works. It is available in the marketplace. Obviously, you can download it and you can test it out. But let me show you how the plugin works and quick demo with that. What I’m going to do here is, so this environment that you’re looking at right now actually does have the plug-in installed. Once you have the plugin installed, you have a couple of options. One is that you can go and take a particular freestyle job and actually convert them or you can actually go and convert an entire folder. You will end up getting an option here saying convert your folder jobs to Pipeline.

I’m going to use this option today. I’m going to click on that. What you get here is that you have a way to name the job. I’m just going to call it Jenkins Plugin Webinar demo. Then, you can actually choose the job that you want to convert. Now, you can see that there are multiple jobs here which are feature jobs that are waiting to be converted. You can start from any point of time. Now, in this particular case these jobs are actually chained together. What that means is that a build job will be followed by a static analysis job, will be followed by an exit job.

There is a chaining of the job, feature job within. You can have any starting point for your conversion. In this case I’m going to use a WebApp-Build. You have an option of saying there, do I want to recursively go and convert all the downstream jobs or do I want to convert one job.

In this particular case, I’m going to convert all the downstream jobs too, which is most of the times what you want to do because you want the plug-in to actually take that burden from you and let it actually convert all the jobs in a way that links it in the final Pipeline too. Right. I’m going to choose this option and then convert it. Now you have another option here where you can actually go and check in the file into your get deposit radar.

You could either get the entire snippet of code created in line or you could actually have a Jenkins code that gets actually checked back into your get deposit radar. That means you post option where you can see the Jenkins file. Let’s do the conversion. You can see the conversion runs really fast. There is no job running here right now. I’m going to just build this job while I show you what the converted job looks like.

If I go to configure for the particular converted job, this job actually is a converted job from the Freestyle one. A lot of the options that you see here have been transposed over from the Freestyle job. If I go all the way down, you can see that a snippet of code got created. This is essentially a groovy script code that got created for you. All of the entries here, for example the node that was chosen, the different stages that you have here, they all got converted over from the Freestyle job.

As an example, you can see that there is a stage called the WebApp-Build which if you recall is the one that I started with for the conversion. If I go further down, you can see that another stage has been created for a unit test and so on and so forth.

The conversion tool is going to automatically link this app for you it’s going to look at a lot of the options that you have in your freestyle job and then convert it into code and ready for you to be used via pipeline.

If I go back again to my job quickly. This is the one that we were running. You can see that it’s already started the process it’s going to continue through all the different stages and complete it for you. This is a quick demo of the tool and how it works when you download it from the Jenkins Plug-in community you’ll see there a lot many other options that you can actually use within the plug-in.

This is a quick snapshot of the Open Source Jenkins Plug-in Community page what it looks like. You can see that there have been quite a few installs — and we released it pretty recently in the marketplace. We have your test running it internally with some of our projects and now we have released it available for you to use within either your Open Source Jenkins Community or even your CloudBees community or CloudBees installation.

A quick slide on what are some of the features that would see available in the plug-ins. As you saw you can either convert a folder of jobs to the pipeline or could also convert a single freestyle job to a pipeline. In most of the cases you would end up doing the former because you have a lot of jobs that needs to be converted. As I mentioned both with Open Source Jenkins and also the CloudBees Enterprise Jenkins, you can further customize it as required.

This is an important aspect because when we work with clients we do often see that the freestyle jobs have a standard– standard tool — the toolchains that are used but they are also very specific new answers they have created for their job. The plug-in has been designed to take care of the lot of common cases for conversion and obviously it cannot cater to all individual customer scenario, given that is the case, we can customize the plug-in further to meet individual client needs. If it is something that is useful for the community at large then always that will get rolled into the Open Source Plug-in Community itself.

The other important aspect that when you’re using CloudBees and using that old based active control the plug-in does adhere to that and make sure any security constraint that you had within your freestyle job is being moved over to the coded pipeline and any access that you had earlier and are not broken, it takes care of that very well. This is an important aspect of what is the scope of the plug-in? As I mentioned earlier, it tries to do a lot of the things automatically for you but it takes your freestyle, notes setup take a freestyle parameter, take most of your standard job and moves them over.

Then, it also allows you to connect to get right now as a repository, it will take some of the environment probably that exist of which in most common properties and then of course, a lot of the bills step in the possible actions. The list of possible actions that production that you see here are all converted for you automatically. We do understand that you may have very specific possible actions very custom to your requirements but those can be done– those will not be done by the plug-in but in cases where those exist, we will put placeholders in your job so that you can actually go in and modify them manually and customize them manually.

After having installed the plug-in, if you’ve used it for hundreds of your jobs even thousand jobs conversion, you will find that it does a good job of converting a lot of your jobs to coded pipeline but often there are cases where you need help either on customizing the plugin for very specific scenarios or you may need help just with training on how to use the plug-in and how to best deploy it within your ecosystem.

Finally, an important aspect of conversion whether it’s manual, whether to the tool is to really test out the converted pipeline and make sure that it is doing what it’s supposed to do. Each of these can be standardized so they can be very custom requirement, Infostretch just provides services to help with all the three.

If you have a scenario where you need either customization or testing or training Infostretch provide very specific services and expertise to help you with that.

Sarah: Great. Thank you so much, Sanil. At this time we would like to get your questions answered and as a reminder you can at any time after the question in the Q&A window at the center of your screen, looks like we have the first question here. Sanil what are the advantages of a pipeline as code?

Sanil: That’s a very good question, the reason why pipeline as code is very important is because once you put in your entire logic for your job as code, you’ve got a lot more flexibility in deciding what the flows should be, what aspects of it should be changed, you’ve got modularity and as I mentioned earlier, a lot of the good software engineering principles that you get out of writing good elegant code you get out of the system, you will get multiple people working on the pipeline. It could actually be– you could have it checked into your source control system and have multiple people work on it together. But these are definitely a great benefit of doing pipeline as code. The flexibility and also having a great elegant architecture.

Sarah: Great Thank you. You have another question here for you. What support is Infostretch committed to provide for the plug-in?

Sanil: Sure. Infostretch as an important practice around DevOps and as part of practice for DevOps, we are very determined to make sure that any tools which need our support and especially the plug-in that we developed is striving and is constantly announced, but for the help for the community. We do have constant support both on the bug fixes and enhancement side and also on any kind of help training side for this plug-in.

Sarah: Perfect. Thank you. And another question here. What percent of jobs can be converted using the plug-in?

Sanil: So if a lot of the jobs are very standard in nature using largely standard tools in the tool chain, typical post-built actions et cetera, most of it can be converted. But in a very typical scenario, what is seen is that you have at least 70% of the jobs that can be converted and there could be another 20% which needs some minor tweaking to be done, and maybe 10% which would need some major customization. So that is a good split to look at when you think about how many can really be converted.

Sarah: Great thanks to Sanil; and I believe that’s all the questions we were able to answer at this time and I just want to thank everyone for joining us today. You will receive an email in the next 24 hours with a slide presentation and link to the webcast replay and if you’ve any questions, please contact us at info@infostretch.com or call 1-408-427-1100 to speak with the representative.

Lastly, we encourage you to download our free plug-in, we listed it right there in the chat box for you. So please, go ahead and do so. Thanks again everyone. Enjoy the rest of your day.