By Yuval Stein, AVP Product Management, TEOCO

When it comes to maintaining and assuring complex telecom networks, CSPs are recognizing opportunities to iterate faster, at lower cost, by utilizing Operations and reducing dependence on more expensive Engineering resources – thanks to automation, analytics, and AI advances in network optimization.

Separation Between Ops and Engineering

Currently, most CSPs have two departments responsible for maintaining and operating their telecom networks, with a clearly defined dividing line between them – Operations teams, consisting of Network and Service Operations Centers (NOC/SOC) and other operational teams, and various Network Engineering units.

Although each CSP may take a slightly different approach, the NOC/SOC teams manage the healing of the network, focusing on short-term, potentially service-impacting issues like alarms and trouble tickets. They control the operational side of field work and perform other immediate-term operational components of the network. In general, they oversee processes that can be standardized, easily repeated, and controlled.

Meanwhile, Engineering teams handle mid to long-term fixes and overall network maintenance. Additionally, Engineering typically supports the NOC/SOC teams as a second or third line of defense when a more complex investigation is required. While both teams are vitally important- they have different costs associated with them. Network Engineers can command more than twice the salary of a Network Operations Analyst, so CSPs need to be mindful to optimize their time.

Roles and Responsibilities are Blurring

With the introduction of analytics and automation tools, traditional operational and engineering roles are being reimagined.

The combination of analytics and repeatable algorithms are starting to turn network optimization use cases that were classically handled by Engineering into short-term activities that can be done more cost-effectively by Networks Operations. Here are two examples:

Onboarding of New Network Functions.  Activities related to the onboarding of new network functions have always been considered, at least until now, to reside fully with Engineering. With some of these functions becoming more automated, “repeatable” activities may now be done and controlled by the Operations side.

Network Maintenance.  The same is true for some of the network maintenance aspects related to network optimization. For example, when network optimization activities can be fully automated through ML algorithms, these can typically be moved to the Operations team – if the processes have matured and been proven to be both reliable and repeatable.

The Case for Shifting Tasks from Engineering to Operations

Tasks don’t have to be zero-touch and fully automated before they start shifting from the engineering side over to ops.  It’s about identifying the repeatable, high-frequency tasks that can be run with limited oversight and getting these off the plate of engineers who need more time to focus on higher level strategic issues- without the constant interruptions of the day-to-day operational environment.

The Ops team can then make sure these automated processes are run at the appropriate times and places and provide the oversight needed to ensure everything is working properly.  Since this is a trainable function, it doesn’t require extensive engineering expertise and the Ops staff can manage these tasks at a lower cost.

Why and when does automation make sense?

Speed.  The resolution (speed) that you can complete a task with automation is obviously much faster than without. Automation allows you to run more targeted activities on a smaller scale, and more frequently. This helps improve the network more quickly while also creating less risk. Since you are only affecting a smaller area of the network, the chances of harming anything are much smaller.

Repeatability.  If a particular optimization activity is relatively stable, you can define a process around it and train someone on how and when to run it. Then the Ops person will only need to go to the Engineer for their expertise if something goes wrong.

The Shift to Operations is Already Happening

When we look back at the area of IT in general, in the past there were distinct divisions between research and design, application development, systems integrators, and professional services. Now, with application development and even R&D being much more automated, operations teams (DevOps) can manage more of the workload.

On the Telco side we see a similar shift.  Some engineering tasks, like resolving standard network problems, are already appearing within the NOC. In fact, many operations teams have created a second or third line of support inside the NOC itself, just for managing these tasks. These are turning into the ‘supervisors of automation’ and have become a new category of worker within the operations department or NOC.  They execute the automated processes, check to see if everything is working well, and then manage and troubleshoot the process when things don’t go as expected.

The new Ops workers will be trained on network automation and how to manage it. They are within operations, but they are more engineering focused – providing a bridge between the two groups, like the DevOps teams in many of today’s IT departments.  These organizational changes are happening gradually. In some cases, these new Ops people may be placed in the organizations somewhere between Engineering and Operations, similarly to what other organizations are doing with DevOps.

A Shift in Tools

As operations and engineering roles emerge and evolve, we’ll likely see Operations and Engineering groups not only sharing similar tasks, but also using the same tools.  The benefit is that this will lead to a tighter alignment of Ops and Engineering – and better designed and optimized networks will be the result.  We’ll also see more functionalities built out within these tools to support this new operations category, and that’s certainly the direction TEOCO is taking with its service assurance product roadmap.

Refocusing software tools around the whole of the CSP, without creating separate “fault-oriented” tools for Operations and “performance data” tools for Engineering, will allow for significant cost savings and help break down the silos that exist today between the two organizations.  The new generation of applications supported by new user interfaces will be those that let you watch the results of the automated algorithms and manage failures or ‘leftovers’ that were not completed by the automated processes. Over time, as successes with these automated operations build upon each other, trust in automated systems and processes, those running them, and the organization as a whole, will increase.

How TEOCO Helps

TEOCO understands the changing needs of CSPs. We’ve added features and functions within the latest release of our Helix Service Assurance Solution Suite, namely Helix Analytics in conjunction with Helix Automation, that specifically address the evolution of Engineering and the larger role Operations teams will play going forward. With powerful machine learning and AI-backed insights that enable processes to be automated and streamlined, both teams can perform at their best, and overhead costs can be optimized.

For more information about how TEOCO can address your evolving service assurance needs, contact us today.