HP Site Flow’s architecture relies on containerization to quickly scale up and down as demand dictates. Mako™ was key to this approach, offering a small memory and resource footprint, without compromising PDF processing performance. Mako also proved its capability and adaptability by rising to HP Site Flow’s development challenges, resulting in increased productivity and profitability for its users.
When HP Inc began developing HP Site Flow, a simple-to-use manufacturing execution system for HP digital press owners, they soon encountered several challenges:
The personalized market was growing
Knowing the standard file format in the industry is PDF, HP based its original solution on open source software. But it quickly outgrew this as new customer requirements were identified, for example to extract metadata or add a barcode; HP needed a more reliable, fully featured solution from a dependable source to manipulate PDFs. Initially they used Adobe® PDF Library. It was satisfactory at first, but they found that it didn’t meet their needs.
Not all PDFs are equal
Personalization also leads to a wide variation in the quality of files entering the system, as consumers use many different tools to create PDFs. HP Site Flow therefore needed to “normalize” the incoming PDFs prior to printing, which was slowing down the production process.
Printing large PDF files at speed
With high volume printing a focus, and the need to print large files, where each file is different, HP Site Flow had to be capable of processing huge PDF files; large documents sent to the printer in PDF format can take a long time to print due to their size and complexity. This can lead to a bottleneck and slow down the printer when all the printer memory is being used trying to render the PDF document in real time.
The ability to scale quicker
The system would need to automatically scale quickly; there may be 100, 10,000 or 100,000 orders to process each day and print jobs must be processed in an efficient and timely way.
HP Site Flow is a manufacturing execution system, available through PrintOS, HP’s cloud-based platform, which is fully integrated with HP Indigo 10000 and HP PWI digital presses.
HP Site Flow makes it easy for print service providers to manage large volumes of orders; it accepts orders, groups them into production batches for printing, then unbatches them so they can be sent to different addresses.
HP Site Flow is focused on the personalized market, where users can handle hundreds of thousands of orders a day.
Andrew Gits, Lead Technical Architect, HP Inc
“We trusted the Global Graphics name because we knew the Harlequin RIP, so we knew the brand is just as trustworthy as Adobe in the print industry.”
HP chose Mako™, an SDK from Global Graphics Software that offers pre-flighting options and streamlines PDFs. Using Mako, HP Site Flow manages the process of taking the PDF and making it suitable for printing. It analyzes the document, working out the consumables required, such as how much toner, paper stock etc.
Andrew Gits, lead technical architect at HP comments: “Mako runs on Linux and it ticked a lot of boxes for us. We got up and running quickly. We built a very lightweight wrapper around it so we could use it from NodeJS at the low level, which we weren’t able to do easily with the Adobe PDF Library.”
Andrew continues: “One of the things Global Graphics did for us early on was to assist with running the system on Alpine Linux. This was a big thing for us because we build everything on containers and so we're deploying containers all the time. The difference between running on something like Ubuntu and Alpine is that Ubuntu is in the hundreds of megabytes per container, whereas Alpine is in the 10s of megabytes, so we could build these much smaller, lightweight containers to do the PDF rendering.” That efficiency allowed HP to reduce container start-up time and run many more of them on a single host, increasing scalability.
Realizing Mako’s potential, HP commissioned the team at Global Graphics Software to implement several new features:
HP wanted to build a PDF viewer in the cloud that didn't rely on having to download the whole PDF. Andrew explains: “Sometimes a customer had a 2GB PDF and if you go into a UI and you're looking at a live preview, you want to be able to zoom in and look at different levels of detail and see different separations etc. We wanted to have that functionality for any PDF in the background. PDFs are now streamed from a url and they dynamically thumbnail.”
Andrew expounds: “It may seem like just another platform, but the move to Alpine Linux had huge implications for us because of how our infrastructure scales. HP Site Flow runs on a platform called Workforce, a dynamically scaling platform that deals with rapidly changing volumes of orders. That means we don't just have four boxes sitting there at all times processing things as they can - we scale up from one to thousands of container-based processes (we call them ‘workers’) running these PDF transformations based on what's currently required, so it's very important the workers are as lightweight, small and efficient as possible to have the capacity to run a thousand of them and then back to one in minutes.”
Andrew Gits, Lead Technical Architect, HP Inc
"I see Mako as a fantastic tool in our toolbox that we can very quickly implement. For example, we’ve recently been looking at PDF complexity - we know we can grab Mako and look at how complex a particular PDF file is. Mako tells us how such complexity will impact our system and what the resulting requirements will be for us to go ahead with that process and we can start to map that. We couldn’t do that without Mako.”
Improved memory usage and CPU performance
David Stevenson, product manager for Mako comments: “One of the things that HP needed us to do was focus relentlessly on performance. For example, by employing innovative caching strategies to improve content reuse and reduce runtime memory usage, we have been able to reduce hours of processing time to just minutes in some cases. This has also made possible processing of huge jobs that would have previously exhausted available memory.
Enhanced Productivity Mode
Together HP and the Mako team also implemented a new feature to automatically evaluate jobs for their suitability for printing using Enhanced Productivity Mode (EPM). EPM is defined as printing color (CMYK) jobs using only three colors (CMY) and generating the Black separation by overlapping the three colors CMY. To determine whether a PDF is a candidate for EPM, Mako applies the same unique color management used for EPM by the RIP, allowing for in-depth examination of the PDF, carefully analyzing different separations, color usage and densities. Andrew continues: “EPM saves hugely on consumable costs and enhances productivity by 33% for colorful jobs. Knowing before a job hits the RIP allows EPM jobs to be managed and charged for correctly. This was a very exciting project and couldn't have happened without Mako. The collaboration between our companies made that possible."
It's not just about Mako
Andrew adds: “It's not just been about Mako, but about the team. The best thing about this project has been working with Global Graphics Software. We could send a message and say we wanted to try and do this thing and very quickly we would get the right people on that call and they would investigate it - and that has been a huge benefit for us. What we've done over the years wouldn't have been possible without that team behind it.”
HP Site Flow is a full-featured manufacturing execution system with high functionality that has resulted in increased productivity and profitability for its users.
Mako now handles 90% of PDF files in the HP Site Flow workflow. Mako analyzes the file, accurately predicting the achievable line speed for every PDF job, optimizing the job for the printer.
Mako’s PDF file optimization process has increased the proportion of jobs that can be RIPped at full engine speed – enabling print service providers to print more jobs, fast, enhancing their productivity.
ABOUT HP GRAPHICS SOLUTIONS PORTFOLIO
HP Graphics Solutions Portfolio offers a suite of tools to brands and print service providers. At the core resides Brand Centre and Site Flow. Site Flow is a manufacturing execution system for Print Service Providers which enables large-scale print production automation and control from job capture to ship. Brand Centre provides publishers, content owners, and PSPs alike a web portal enabling easy order ingest and tracking for customers.
On top of these services are several further tools including Asset Locker and AutoFlow. Asset Locker being a digital asset management and proofing solution while AutoFlow provides a simple UI to automate complex or tedious file preparations and reduce the labor, cost, and time associated with them.
From order management through to printing and shipping, the HP Graphics Solutions portfolio provides the means to optimize and profit through every step of the print manufacturing process.