Splunk Enterprise Security v8.0.2 is out since January, 22nd.
I will present you a new feature introduced by this version : the detection versioning.
Awaited for a long time, this functionality is interesting in several manners.
First of all, in theory, you can avoid using any git or gitlab tool to keep versions of your detections. It is directly integrated in ES.
Secondly, it makes troubleshooting detections easier : all you have to do is to switch from one version to another directly from the UI.
Limitations
Before starting, there are limitations to the versioning feature provided with ES 8.0.2.
Content versioning
Only detections provided in Enterprise Security Content Update (ESCU) and ES are imported. It does not import nor authorizes user-created content to be manageable by Versioning. But you can still move your own detection rules to Enterprise Security app before the versioning activation, it will work.
Source: Create multiple versions of a detection in Splunk Enterprise Security – Splunk Documentation
Versioning deactivation
If you activate the versioning, you cannot deactivate it from the GUI. The documentation asks you to contact Splunk support if you want to deactivate it.
You have been warned !
Index creation
Activating the versioning feature creates an index (named cms_main). Be advised of that before turning it on, or plan where this index must be created.
Let’s go for the process to activate and use this feature.
The Process
By default, versioning is not activated. You need to activate it by yourself by going to ES / Configure / General Settings and “Turn on” on the Detection Version tile:
It might take up to 10 minutes to turn on, as indicated in the documentation : Use detection versioning in Splunk Enterprise Security – Splunk Documentation
This process activates the SA-ContentVersioning app located in Splunk directory, uses the cms_main index (by default in $SPLUNK_DB directory) where changes will be stored.
It also adds a lookup collection called ‘cms_metadata’ which will contain metadata information, such as the id of every detection rules using versioning, its hash, the version info, the publishing time, the user modifying the detection, the app it belongs and the parent version (only the number, not the content) of that detection.
All detections included in Splunk ES and ESCU (if installed) are automatically imported and manageable by the versioning.
And then you get a notification on the upper right corner :
Additional information : Create multiple versions of a detection in Splunk Enterprise Security – Splunk Documentation
Usage
After versioning is turned on, you can find additional information in the “Version” column in the Content Management.
Now if you edit a versioned detection, you will find additional options in the Edit page:
On the right in the Details panel, you get an ID for your detection (called detection_id) and versions panel on the bottom. The version in green is the activated one.
You now have the button called “Save as new version” instead of a simple “Save” button.
If you modify something on the rule, a new version appears after the saving :
By default, the new version has “Off” as Status.
If you want to activate that version, you must click on that new version (the version appears at the end of the URL):
Then you must select “On” at the bottom. It’s activated immediately (no need to save).
Then the version is activated :
At the upper right corner you get a notification indicating the new version is activated:
Back to Content Management, you can find, at the end of each detection rule that has a version, the Action menu that can show you the Version history in a new modal window :
The modal window :
Unfortunately, there is now way to view the differences between these versions : if you click “View”, you get to the Edit page of that detection version.
Additional information
The versioning information is present in multiple locations.
You have the cms_metadata lookup, the cms_main index, the savedsearches.conf file (see below), the content_versioning.conf, feature_flags.conf and content-version.conf files. These file do not have any documentation now on Splunk.com but you can find the spec files in the SA-ContentVersioning/README app directory.
content_versioning.conf: contains the[content_versioning]stanza, referencing all allowed apps that are included in content versioning. Only ES and ESCU detection rules are referenced here. This parameter must be in JSON format.feature_flags.conf: contains 2 parameters,versioning_activatedandversioning_initbut are not documented. The first one is related whether versioning is activated or not.
Additionally, the savedsearches.conf file contains, for every detection, a specific parameter named “action.correlationsearch.metadata“, stored in JSON format and containing version information:
action.correlationsearch.metadata = {"detection_id": "f235c3dd-26c5-405f-ac7b-671dff1f6640", "detection_version": "1", "minor_version": 1, "version": "1.1", "parent_detection_version": "", "app_version": "8.0.2", "source_id": "", "source_detection_name": "", "version_hash": "0b22f572fb8c44f7f21c1b385fed632308452031aa5cd81b6d478edf0c9a5307", "create_time": 1738681838.650662, "publish_time": "", "user": "", "notes": "[]", "deprecated": 0, "app_name": "SplunkEnterpriseSecuritySuite", "_key": "0b22f572fb8c44f7f21c1b385fed632308452031aa5cd81b6d478edf0c9a5307"}
The only way to get your own detection rules versioned with this version of ES is to put your rules in one of following allowed-apps : DA-ESS-AccessProtection,DA-ESS-EndpointProtection,DA-ESS-IdentityManagement,DA-ESS-NetworkProtection,DA-ESS-ThreatIntelligence,SA-AccessProtection,SA-AuditAndDataProtection,SA-EndpointProtection,SA-IdentityManagement,SA-NetworkProtection,SA-ThreatIntelligence,SA-UEBA,SA-Utils or SplunkEnterpriseSecuritySuite.
The detection_id in these metadata is created by the detection versioning feature. And version 1.1 of every rule is the original version of the detection when the feature is turned on.
Conclusion
I’m pretty skeptical on this detection versioning feature on Splunk ES. I thought all detection rules would be eligible to be versioned, but it is restricted to specific apps. That’s the first point.
And when you have a versioning tool, you guess that you can do/view a simple diff between two versions of your detection rules.
So the fact that this feature exists now is a good thing. I’m simply hoping that it will improve and simplify over time. The usage of lookup, index, specific and non-documented files and current restrictions (limited to ES/ESCU apps, no deactivation from GUI, no diff, no built-in dashboard) lets the preference go to github/lab/ops or Logcraft to manage your detection rules.