Splunk ES 8.0.2 – Versioning feature review

Table of contents

1. Limitations
   • Content versioning
   • Versioning deactivation
   • Index creation
2. The Process
3. Usage
   • Additional information
4. Conclusion

Splunk Enterprise Security v8.0.2 is out since January, 22^nd.

I will present to you a new feature introduced in this version: detection versioning.

Long awaited, this functionality is interesting in several ways. First of all, in theory, you can avoid using Git or GitLab to keep versions of your detections, as it is directly integrated into ES.

Secondly, it makes troubleshooting detections easier: all you have to do is switch from one version to another directly from the UI.

Limitations

Before starting,  there are limitations to the versioning feature provided with ES 8.0.2.

Content versioning

Only detections provided in Enterprise Security Content Update (ESCU) and ES are imported. User-created content is neither imported nor authorized to be managed by Versioning. However, you can still move your own detection rules to the Enterprise Security app before enabling versioning, and it will work.

Source: Create multiple versions of a detection in Splunk Enterprise Security – Splunk Documentation

Versioning deactivation

If you enable versioning, you cannot disable it from the GUI. The documentation advises contacting Splunk support if you want to disable it.
You have been warned !

Index creation

Enabling the versioning feature creates an index namedcms_main. Be aware of this before enabling it, or plan beforehand where the index should be created.
Let’s go for the process to activate and use this feature.

The Process

By default, versioning is not enabled. You need to enable it manually by going to ES > Configure > General Settings and selecting “Turn on” in the Detection Versioning tile.

This process enables the SA-ContentVersioning app located in Splunk directory and uses the cms_main index (by default in $SPLUNK_DB directory) to store changes.

It also adds a lookup collection called ‘cms_metadata’ which contains metadata for each detection such as:

• ID
• hash
• version information
• publishing time
• user modifying the detection
• app it belongs to
• parent version (only the number, not the content)

All detections included in Splunk ES and ESCU (if installed) are automatically imported and managed by versioning.

And then you get a notification on the upper right corner :

Additional information : Create multiple versions of a detection in Splunk Enterprise Security – Splunk Documentation

Usage

After versioning is turned on, you can find additional information in the “Version” column in the  Content Management.

Now if you edit a versioned detection, you will find additional options in the Edit page:

On the right in the Details panel,  you get an ID for your detection (called detection_id) and versions panel on the bottom. The version in green is the activated one.
You now have the button called “Save as new version” instead of a simple “Save” button.
If you modify something on the rule, a new version appears after the saving :

By default, the new version has “Off” as Status.

If you want to activate that version, you must click on that new version (the version appears at the end of the URL):

Then you must select “On” at the bottom. It’s activated immediately (no need to save).

Then the version is activated :

At the upper right corner you get a notification indicating the new version is activated:

Back to Content Management, you can find, at the end of each detection rule that has a version, the Action menu that can show you the Version history in a new modal window :

The modal window :

Unfortunately, there is now way to view the differences between these versions : if you click “View”, you get to the Edit page of that detection version.

Additional information

Versioning information is stored in multiple locations:

• cms_metadata lookup
• cms_main index
• savedsearches.conf (see below)
• content_versioning.conf
• feature_flags.conf
• content-version.conf

These files are not currently documented on https://docs.splunk.com/, but you can find their spec files in the SA-ContentVersioning/README app directory. Lets see what information these artifacts contain.

• content_versioning.conf : contains the [content_versioning] stanza, referencing all allowed apps that are included in content versioning. Only ES and ESCU detection rules are referenced here. This parameter must be in JSON format.
• feature_flags.conf: contains 2 parameters, versioning_activated and versioning_init but are not documented. The first one is related whether versioning is activated or not.

Additionally, the savedsearches.conf file contains, for every detection, a specific parameter named “action.correlationsearch.metadata“, stored in JSON format and containing version information:

action.correlationsearch.metadata = {"detection_id": "f235c3dd-26c5-405f-ac7b-671dff1f6640", "detection_version": "1", "minor_version": 1, "version": "1.1", "parent_detection_version": "", "app_version": "8.0.2", "source_id": "", "source_detection_name": "", "version_hash": "0b22f572fb8c44f7f21c1b385fed632308452031aa5cd81b6d478edf0c9a5307", "create_time": 1738681838.650662, "publish_time": "", "user": "", "notes": "[]", "deprecated": 0, "app_name": "SplunkEnterpriseSecuritySuite", "_key": "0b22f572fb8c44f7f21c1b385fed632308452031aa5cd81b6d478edf0c9a5307"}

The only way to get your own detection rules versioned with this version of ES is to put your rules in one of following allowed-apps : DA-ESS-AccessProtection,DA-ESS-EndpointProtection,DA-ESS-IdentityManagement,DA-ESS-NetworkProtection,DA-ESS-ThreatIntelligence,SA-AccessProtection,SA-AuditAndDataProtection,SA-EndpointProtection,SA-IdentityManagement,SA-NetworkProtection,SA-ThreatIntelligence,SA-UEBA,SA-Utils or SplunkEnterpriseSecuritySuite.
The detection_id in these metadata is created by the detection versioning feature. And version 1.1 of every rule is the original version of the detection when the feature is turned on.

Conclusion

I’m pretty skeptical on this detection versioning feature on Splunk ES. I thought all detection rules would be eligible to be versioned, but it is restricted to specific apps. That’s the first point.
And when you have a versioning tool, you guess that you can do/view a simple diff between two versions of your detection rules.
So the fact that this feature exists now is a good thing. I’m simply hoping that it will improve and simplify over time. The usage of lookup, index, specific and non-documented files  and current restrictions (limited to ES/ESCU apps, no deactivation from GUI, no diff, no built-in dashboard) lets the preference go to github/lab/ops or Logcraft to manage your detection rules.