Splunk Engineer Senior - TS/SCI, Washington, DC

M&S Consulting was conceived in 2002 with the vision of creating highly effective teams of elite consultants to deliver strategic process and technology solutions to enterprise organizations across the US. Our commitment to delivery in complex environments and long-term customer success has merged process and technology into innovative solutions, established deep pockets of expertise, and enabled innovative transformation for evolving businesses.

We have intentionally cultivated steady growth focused on being approachable and helpful to our dearly valued clients and closely cared-for employees. M&S people simply “care hard”, and this reflects in our work products, our interactions, and our culture.

M&S Consulting is seeking a  Splunk Engineer (Senior) to work in Washington, DC. Candidate must possess an active Top Secret/SCI Eligible.  

Job Description:
Possesses and applies a comprehensive knowledge across key tasks and high impact  assignments. Plans and leads major technology assignments. Evaluates performance results  and recommends major changes affecting short-term project growth and success. Functions  as a technical expert across multiple project assignments. May supervise others. Minimum  of 8 years’ experience recommended. In absence of years of experience, certifications or past  work may be used to show the level of experience needed to perform at this level.

Job Requirements:

Minimum of 4+ years of progressive experience in managing Splunk and integrations to Splunk (index's/forwarders, etc). Three years of experience in writing Splunk queries is required.

• Minimum  of 8 years’ experience overall recommended.
• In absence of years of experience, certifications or past  work may be used to show the level of experience needed to perform at this level.
• Supervise, coordinate and/or design, develop and perform additions and changes to network  hardware, infrastructure server environments, and operating systems, and attached devices,  including investigation, analysis, recommendation, configuration, installation, and testing of new  network hardware and software.
• Provide direct support in the day-to-day operations on network  hardware and operating systems including the evaluation of system utilization, monitoring response  time and primary support for detection and correction of operational problems.
• Troubleshoot at the  physical level of the network, working with network measurement hardware and software, as well as  physical checking and testing of hardware devices at the logical level working with communication  protocols.
• Participate in planning design, technical review and implementation for new network  infrastructure hardware and network operating systems for voice and data communication networks.  
• Perform systems management and integration functions.
• Exhibiting proficiency with virtualization  technologies and network administration.
• Maintain network infrastructure standards including  network communication protocols such as TCP/IP.
• Provide technical consultation, training and  support to IT staff as designated by the government.
• Diagnose and resolve complex communication  problems. Perform systems management and integration functions.
• Responsible for design,  deployment, evaluating, testing, certification, patching, addressing interoperability issues, not only  of the base storage & backup system, but for all features, components, and application  dependencies.
• Ensures systems are current with approved patches and assists with certification and  system accreditation tasks, composes SOPs, and Quick Reference guides for users.
• Provides top level technical expertise responsible for design and development of a client/server environment.  
• Develops strategy of client/server system and the design infrastructure necessary to support that  strategy.
• Advises on selection of technological purchases with regards to processing, data storage,  data access, and applications development.
• Sets standards for the client/server relational database  structure for the organization. Advises of feasibility of potential future projects to management.
• Proficient in designing, implementing, and managing Splunk environments, including cluster architecture.
• Advanced knowledge of Splunk Search Processing Language (SPL)
• Expertise in developing complex queries, dashboards, and reports using SPL.
• Ability to optimize searches for performance and scalability.
• Strong experience in data onboarding, parsing, and indexing, including log management and extraction of meaningful metrics.
• Familiar with various data input methods like Syslog, HTTP Event Collectors (HEC), and APIs.
• Knowledge of Splunk Enterprise Security (ES) and/or IT Service Intelligence (ITSI)
• Experience in configuring and managing Splunk ES and /or ITSI modules.
• Understanding of security information and event management (SIEM) concepts.
• Proficiency with REST APIs and other scripting tools (e.g. Python, Bash) to automate tasks and integrations.
• In-depth knowledge of Linux/Unix systems administration, including shell scripting and performance tuning.
• Familiarity with cloud platforms (AWS and/or Azure) and experience managing Splunk in cloud environments.
• Ability to troubleshoot and resolve Splunk performance issues.
• Experience with scaling Splunk in large environments and ensuring high availability

Preferred Requirements:

• Certifications in cyber security logging and data analysis are preferred.

*M&S Consulting proactively fulfills its role as an equal opportunity employer. We do not discriminate against any employee or applicant for employment because of race, color, sex, religion, age, sexual orientation, gender identity and expression, national origin, marital status, physical or mental disability, status as a protected veteran, or any other characteristic protected by law.