Workload automation isn’t just a series of lengthy tasks (job scheduling) performed at 3 a.m. in the backrooms of IT departments anymore. It’s got a new look and feel, and can significantly assist the enterprise via:
- Self-Service Integration Options: Big Data and Cloud-based applications may be the future of data driven processes, but they are difficult and time-consuming to integrate with additional sources and tools (such as Business Intelligence) without self-service automation. These automation solutions reduce the need for IT and facilitate ad-hoc, on-demand integration that can correlate to all of the different technologies relevant to the enterprise today. The trend towards self-service is aided by recent innovations in scriptless automation.
- Workload Execution Spikes: With 24/7 access to products and services routinely enabled via mobile and web-based applications, organizations no longer know when their workloads may peak and reach the point of capacity – without automation tools. Certain tools can now presage such spikes and even mitigate them by redistributing workloads accordingly, substantially decreasing the potential for delays and system and server overloads. This capability may prove invaluable with increasing usage of Big Data and its unpredictable speeds and scalability requirements.
- DevOps: Automation tools can provide a crucial means of integrating test environments for IT operators and development teams, increasing time to use for new applications and smoothing out the entire process.
IT departments and developers are still the primary customers for workload automation solutions. However, this fact will likely become challenged in the coming months by the assortment of self-service options available. The principle driver for this tendency is the disparate amount of technologies, tools, and platforms that the enterprise has readily adopted. Whether these various software and hardware products relate to Big Data, analytics, MDM, or CRM, it is still necessary to get them to integrate at varying points to realize their full potential. Presently, various technologies and platforms have their own respective options for job scheduling, but require an outside source to integrate with one another.
With self-service automation, business users are able to dictate the pace of integration. IT still plays a pivotal role in setting up these tools and integrating them with different components, but the business controls when such integration takes place and how it immediately affects its particular objective. Instead of waiting overnight for standard integration scheduling, the business can determine when to combine sources for more readily available insight. In this respect, automation tools are very much aligned with the general movement in IT (encompassing Big Data, Cloud Computing, and Business Intelligence) that gives end users the data they need quickly to make decisions.
Colin Beasty of Advanced Systems Concepts, which offers a workload automation and job scheduling tool ActiveBatch, discussed the impact of self-service automation on the enterprise.
“Because the business is asking for more quicker, the IT organization is turning to automation. They can’t rely on all of the platform-specific scheduling tools that are out there, so workload automation is becoming a really big player in terms of being able to integrate and automate all of these disparate data sources and integration tools to more efficiently push that data downstream to the BI tools.”
Actual Use Cases
Whereas conventional workload automation relied upon lengthy scripts which accounted for some of the delay in IT processing requests, self-service tools are carefully designed by vendors with templated steps so that processes can become automated much faster. One of the common needs for automation is the transfer of files via File Transfer Protocol (FTP). ActiveBatch and other products can give users views of different workloads and allow them to act on them in real time. Within the finance industry, for example, end users can track down transfers and payment information in close to real time without unnecessary steps. Beasty described such a situation with a customer.
“I was speaking to a customer a week ago that actually allows their accountants access to ActiveBatch by its web-based console,” Beasty said. “In the past, if they didn’t get a payment they would have to call up the help desk and IT operations to track it down hours later. Now, the first thing they do is execute through ActiveBatch and the FTP comes through without any calls.”
Automation is valuable for repetitive processes and tasks that are time consuming to complete. Within the education industry, universities are looking to automate the process of delivering student course loads and scholastic resources through virtual machines (VMs). Whereas many educational institutions take a flat file of student information, parse it and provision it to individual students manually, workload automation products can automate this process and reduce the strain on administrative resources.
“Once they get the flat file, that can serve as the trigger to have ActiveBatch automate it by taking the information, give them each a virtual machine, the login credentials, and email the material to students who are ready to go by the first day of class,” Beasty said.
This same concept applies to the administrative on-boarding process of organizations, and can serve to both provision and de-provision employees.
Workload Execution Spikes
In addition to integration and expediting the process of repetitive tasks, workload automation solutions can also help to regulate workloads and prevent spikes in usage that could potentially cause system and server overloads. Most tools have some execution agents with predictive capabilities in which, based on the level of workloads at the present, they are able to foresee whether those levels will likely escalate or remain non-threatening. These agents are extremely useful for connecting with Cloud resources to determine when additional scalability is required – as may be the case with Big Data or with Big Data analytics.
In these instances automation tools can systematically provision workloads onto other machines in a proportional way so that none of them reach the point of capacity. According to Beasty:
“Up until recently, workload automation tools have focused on monitoring only the workloads that they are executing on a machine. Now execution agents are measuring everything a machine is doing – because it could be doing other things as well – and if it’s getting to that threshold where something’s going to go boom, there are algorithms that can throttle those workloads across other machines.”
One of the most widely used aspects of integration for which automation has repeatedly been used is that between IT operators and the development team. The term DevOps was coined fairly recently and has figured prominently in organizations for a number of reasons – not the least of which is the Big Data phenomenon and the analytic insights it yields, which directly affects new products and services companies want to market.
It is not uncommon for these two departments to use the Cloud to work together; they can increase their uniformity and push past the testing and development phase to get goods to the market faster by integrating their processes with workload automation. The number of organizations doing just that has significantly expanded in 2013, and should continue to do so in the coming year and beyond.
Beasty commented on this trend.
“There’s going to be processes that are integrated that span the different tools that IT uses and the different tools that the development team uses. In the past, organizations were stuck with this catch 22 where they’d want to be able to automate some of this so the teams could work faster and work in unison, but they would have to do a lot of scripting to try to automate those processes across the disparate platforms. Ultimately, that’s why they choose workload automation tools.”
The Question Remains
The benefits of workload automation are clear – it furthers the growing movement to empower the business, manages workloads efficiently and in some cases predictively, and is integral to the development of new products and services based on some of the most salient technological trends today (Big Data, the Cloud, analytics).
But what about the employee whose job functions have now become automated?
“We have 600 customers; I haven’t talked to all of them but I’ve talked to a lot of them,” Beasty acknowledged. “No one has ever gotten fired because of automation. Typically what people automate are the repetitive things that take place on a daily basis, so it’s not about doing more with less—it’s about doing more with the same.”
Which is the same as simply doing more.