KEY ACCOUNTABILITIES
CUSTOMER
- Perform controlled resolution of production incidents and problems, including investigation, prioritization, troubleshooting, escalation and service restoration, working with business and technology partners as appropriate.
- Provide effective day-to-day L2 production application support and maintain application/system availability at or above established service levels while minimizing operational risk and customer impact.
- Participate in a scheduled after-hours/on-call rotation supporting business-critical applications, including evenings, weekends and holidays as required.
- Respond to production alerts and escalations while on call, perform initial investigation and troubleshooting, and remain engaged through service restoration, appropriate escalation or operational handoff.
- Investigate production issues using application and system logs, monitoring tools, operating-system diagnostics, database queries and other available technical information.
- Analyze application logs and monitoring information to identify failures, error patterns, performance issues and upstream/downstream dependencies.
- Perform approved application recovery activities such as service restarts, application restarts, job reruns, process recovery and other documented operational procedures as required.
- Perform the L2 incident management role for outages and high-priority incidents, including acting as investigator, communicator, tracker, escalator and driver through service restoration.
- Participate in high-priority/service-impacting incident bridges and provide clear technical findings, actions taken, current status and escalation requirements.
- Ensure timely notification and escalation of production issues and engage application development, infrastructure, database, middleware, network, vendor or other technology teams when additional expertise is required.
- Maintain accurate incident documentation, including investigation performed, technical findings, recovery actions, escalations and resolution details.
- Provide clear handoff of unresolved incidents, ongoing investigations and production risks between shifts or support resources.
- Identify recurring incidents and error patterns and support Problem Management and root-cause analysis to reduce repeat production issues.
- Continuously improve production stability by partnering with key stakeholders to maintain, monitor and enhance supported applications and systems.
- Support state-of-health monitoring, application availability, SLA maintenance and monthly/scheduled maintenance activities.
- Partner with relevant teams to schedule, package and release application changes while minimizing implementation time and production impact.
- Schedule and implement changes to supported components in accordance with approved Change Management procedures, including testing, stakeholder signoff, validation, monitoring and rollback where required.
- Support scheduled after-hours deployments, maintenance activities, application validations and production changes as required.
- Identify and address application, integration and data issues that affect application integrity, availability or performance.
- Develop an understanding of application architecture and technical dependencies to support effective troubleshooting and provide guidance to development and technology partners.
- Interact with clients and business partners to provide quality service and timely communication during production issues and service outages.
- Work with third-party vendors as required to investigate issues and fulfil production support requirements.
SHAREHOLDER
- Monitor supported applications and systems using meaningful operational metrics to identify availability, performance, capacity and stability concerns.
- Use enterprise monitoring and log-analysis tools such as Dynatrace, Splunk or equivalent technologies to proactively identify and investigate production issues.
- Assess and analyze opportunities to improve application stability, monitoring, operational performance and resource utilization.
- Identify monitoring and alerting gaps and work with appropriate stakeholders to improve early detection of production issues.
- Assess and analyze optimization opportunities to the operational environment to improve performance and/or resource utilization
- Ensure effective change management discipline is use
- Adhere to existing processes/standards, business technology architecture, risk and production capacity guidelines; plan, monitor and escalate issues as required
- Follow standards, policies and procedures to ensure compliance with the Disaster Recovery Plan (DRP) and applicable Business Recovery Plans (BRP)
- Identify/implement process improvements to enhance revenue, customer experience and/or reduce costs
- Comply with well-defined enterprise technology delivery practices and standards and project management disciplines
- Make effective use of the cost management processes in place in own unit
- Continuously enhance knowledge/expertise in TD services, applications, infrastructure, analytical tools and techniques that can contribute to effective solution development/delivery
- Keep current with industry and/or business trends
- May perform testing according to test plans, monitor and report on results, and work with others on problem resolution
- As required, support the development of business cases, RFI/RFP and service level agreements with vendors/suppliers consistent with IT requirements/guidelines
EMPLOYEE / TEAM
- Work effectively as part of the application support team, supporting other team members in investigating and resolving critical production issues.
- Participate fully in the team's on-call rotation and maintain the system access, technical knowledge and availability required to independently respond to production incidents.
- Prioritize and manage workload to deliver quality results and meet incident, request, problem, change and SLA timelines.
- Communicate production issues, technical findings, risks and escalation requirements clearly and in a timely manner.
- Provide effective operational handoffs for active incidents, ongoing investigations, scheduled changes and outstanding production risks.
- Participate in knowledge transfer within the team and business units
- Identify and recommend opportunities to enhance productivity, effectiveness and operational efficiency of the business unit and/or team
BREADTH & DEPTH
- Performs hands-on L2 support across all aspects of assigned applications and systems.
- Supports moderate to highly complex production applications with minimal management supervision.
- Independently investigates production application incidents using monitoring tools, application logs, Linux/Unix diagnostics, database queries and other available technical evidence.
- Able to independently perform initial production troubleshooting during on-call periods and determine when additional application, infrastructure, development or vendor escalation is required.
- Actively involved in resolving customer-impacting and service-impacting application issues.
- Performs structured problem determination, including technical data gathering, log analysis, event correlation, troubleshooting and identification of probable cause.
- Understands application architecture and upstream/downstream dependencies sufficiently to determine appropriate troubleshooting and escalation paths.
- May develop scripts using appropriate tools to address gaps in the application/system, to simplify and enhance the application/system environment
- Performs problem determination including data gathering and analysis
- Generally reports to a Manager or Senior Manager
EXPERIENCE & EDUCATION
- 3–5 years of hands-on production application support experience in a complex enterprise environment.
- Demonstrated experience troubleshooting and resolving production application incidents, including high-priority/service-impacting incidents.
- Experience participating in an after-hours/on-call support rotation.
- Strong working knowledge of Linux/Unix, including command-line troubleshooting, process investigation, file systems, CPU/memory utilization and application logs.
- Experience with enterprise monitoring/log-analysis tools such as Splunk and Dynatrace.
- Experience with ServiceNow Incident, Problem and Change Management.
- Ability to analyze logs, identify error patterns, correlate events and determine appropriate application, infrastructure or downstream escalation.
- Understanding of application/middleware technologies such as WebSphere, JBoss, Spring Boot, Java or equivalent enterprise application platforms, where applicable.
- Working knowledge of SQL for application and data troubleshooting.
- Experience supporting production deployments, change implementation, validation and rollback.
- Ability to independently manage an incident from initial alert through investigation, escalation, service restoration and documentation.
- Experience with root-cause analysis, recurring incident identification and Problem Management.
- Scripting/automation experience using PowerShell, shell scripting or similar tools is an asset.