Read the UC4 Blog 
Customer Snapshot
Company: DoubleClick
Business/Industry: Digital Marketing
Company Headquarters: New York, New York City, USA
Webpage: www.doubleclick.com
UC4 Customer: since 1999
Technical Environment: misc. Interfaces
The Company
DoubleClick, the world's leading digital marketing solutions company, is saving hundreds of hours per month in network operator time by using an enterprise job scheduler to automate scheduling for billions of batch and maintenance jobs every day. The company helps online marketers and publishers deliver more than 2 billion impressions per day through its ad management system, DART (Dynamic Advertising Reporting Targeting).
Each of these transactions requires a separate accounting process that is handled by its custom accounting feeder system. This system requires the execution of a large number of batch and maintenance jobs, such as performing system maintenance on more than 600 Windows NT servers nightly. To help control this environment, DoubleClick invested in an enterprise scheduling solution from UC4 Software that automates both Unix and Windows NT/2000 jobs using a centralized management console.
"The bottom line is that we have improved the efficiency of our batch processing while saving at least 600 hours per month, probably a good deal more", said David Mitchell, senior automation engineer for DoubleClick. "That is equivalent to at least three full time positions. The scheduling software provides a single point of management for hundreds of different batch and maintenance procedures, freeing operators from mind-numbing job execution and monitoring tasks. The product has made it possible for us to continue our aggressive growth and manage an ever increasing work load without additional staff."
The Challenge
DoubleClick's DART is the leading Web-based ad management solution, providing Web publishers, agencies and merchants with comprehensive inventory management, precision targeting, ad delivery, and sophisticated campaign performance reports. As the leading ad management service, DART currently serves an average of 2 billion ads daily worldwide. Running an ad management system of this magnitude presents an enormous financial reporting challenge especially since DoubleClick bills clients according to the number of ads that are actually delivered to customers. To accurately bill its customers, the company must track the billions of impressions that it serves up every day through DART.
DoubleClick utilizes an enterprise resource planning solution within a UNIX environment to manage financial accounting. However, a considerable amount of data processing is required before the transactions can be imported by this system. A large number of batch occur to move the transactions from one step in the process to the next, as well as many repetitive maintenance operations that need to be performed.
Previous scheduler
"Automating these jobs was a lot harder than it might look at first glance", Mitchell said. "This was not just a simple matter of scheduling jobs to run at certain times. Very complicated dependencies are involved. For example, long sequences of jobs must run in exactly the right order and additional sequences can run only when certain upstream sequences are completed. If anything is kicked off out of sequence, major problems are almost certain to occur. Potentially a whole series of jobs could run without the required conditions having been fulfilled. That could result in having to rerun them all by hand or we could generate bad data that would take a considerable amount of time to clean up."
The Solution
As the volume on the company's network expanded and new services were added, the amount of time required for executing batch jobs continued to grow. Mitchell evaluated three leading scheduling tools and selected the UC4 Enterprise Job Scheduler. He chose the UC4 platform because it can track complex interdependencies from a central console as well as automatically alert operators in case of a job failure.
"The UC4 master/agent relationship makes it possible to control all of our processes within a single schedule", Mitchell said. "I was impressed with the strong load balancing capabilities and the fact that the graphical interface eliminates the need to learn a scripting language. I created a few test chains on a demo script that indicated the program could handle just about any type of interdependencies."
Within a short period of time, Mitchell had developed several 'chains', the UC4 term for schedules that handle a series of jobs with multiple dependencies. The UC4 platform enables operators to balance loads by specifying the number of threads available for a chain, and keeps these threads and only these threads fully utilized until the chain concludes.
The Results
The first few chains on the demo script executed perfectly, automatically scheduling the jobs that used to be executed manually while managing all dependencies at the same time. If a job failed, the scheduler will not begin execution of any jobs that are dependent on the first program and instead immediately fires off a message to a staff member to fix the problem. An email will typically go to the person in charge of the job in question and, in the case of important jobs, a page may be fired off as well. This means that rather than having to get involved in each and every batch job, operations staff only deals with the exceptions, typically a job or two per night. Once they address the problem, operators simply re-execute the chain and the scheduler resumes control over batch processing.
"Based on the success of our original implementation we have continued to expand the UC4 platform to cover a wide range of new processes, such as our permission-based email marketing system", says Mitchell. "Every week that goes by we find more processes that can be automated using the UC4 scheduler. It should come as no surprise that management considers this project to be a resounding success."
"The bottom line is that we have improved the efficiency of our batch processing while saving at least 600 hours per month, probably a good deal more. That is equivalent to at least three full time positions. The scheduling software provides a single point of management for hundreds of different batch and maintenance procedures, freeing operators from mind-numbing job execution and monitoring tasks. The product has made it possible for us to continue our aggressive growth and manage an ever increasing work load without additional staff."
David Mitchell, Senior Automation Engineer
DoubleClick, USA
