Deep Dive into Writing Back to Axonius (Part 3 – Custom Enrichment)
Introduction
On my last month’s article, we held the discussion regarding the Data Enrichment fields and how they are used and how to enable them. This article can be found here: https://support.axonius.com/hc/en-us/community/posts/4409146455191-Deep-Dive-into-Writing-Back-to-Axonius-Part-2-Data-Enrichment-
The month prior the article I wrote covered the difference between tags and custom fields. You can find that article here: https://support.axonius.com/hc/en-us/community/posts/4407260908055-Deep-Dive-into-Writing-Back-to-Axonius-Part-1-Tags-vs-Custom-Data- (If you like it, please leave a message or upvote)
This month, we are talking about Custom Enrichment. Custom Enrichment is slightly different from Data Enrichment as we can build out MASSIVE date enrichments for a specific adapter.
- What is Custom Enrichment?
- How is it different from Data Enrichment?
- How is it different from Custom Fields?
- Enrichment Statement
- More Nuance info
- FAQ’s
What is custom enrichment?
Custom enrichment is a function of the Axonius system where a customer takes custom data that is determined by if an asset is in a subnet, or something in the adapter contains or equals a key word to what is noted on the CSV index and adds that data to the adapter data set. When the user uploads a CSV with the key field as well as enrichment outputs, the key fields will connect to the linked fields and provide key attributes that can be queried on or used in enforcements. This is a simple to scale process that will give you a large amount of data back to your adapter input if you so choose to add data that is meaningful to you.
How is it different from Data Enrichment?
Data enrichment has most of the functionality that Custom Enrichment has, except Data enrichment is meant to update the “network interface fields” in the aggregated fields and not touch anything with the adapter level data structures.
How is it different from Custom fields?
While custom fields do touch on many of the same types of attributes that are found in the system, once again, these either create updates to aggregated fields OR creates a functional 3rd party field that can be as dynamic as Custom Enrichment fields but sits as a separate data adapter in the form of Custom Fields. The other key difference between Custom Fields and Custom Enrichment is that Custom Enrichment can be loaded by CSV and can look up a contains, equals or within subnet on the CSV input, whereas all of these things can be done in the query wizard for Custom Fields, HOWEVER, it cannot be done to scale so each rule is done in a single fashion and must be updated as such. Custom Fields can be difficult to scale for large rule sets. The roadmap is projected to release folders; this will allow for much easier organization of these rules.
This feature can be found in the settings icon (top right hand corner, there is a sprocket that says “system settings” if you hover over it) > go to the Global Settings Tab > scroll to the Custom Enrichment (BETA) section.
Enrichment statement
Axonius documentation does a pretty good job of walking the user through the enrichment statement. If you want to check it out. Click here: https://docs.axonius.com/docs/custom-enrichment?highlight=custom%20enrichment
There are a few small nuances to be aware of and they are not prevalent in the documentation. First, see below for the enrichment statement. When a customer would like to enrich the JAMF adapter with information found in the type and distribution pages, they may write the following statement:
enrich 'devices' with (kernel,date_annouced,released_date) on (source.Type_and_Distribution in device.jamf_adapter.[OS: Type and Distribution])
In this exercise,
The statement breakdown would be as follows:
- Enrich ‘Type’ (enrich 'devices')this references that you are enriching the devices table
- “with (kernel,date_annouced,released_date)” This represents the fields that you want to bring in from the CSV.
- on (source.Type_and_Distribution) (“source.” is the key added with the name of the field you are matching with. field is “id”. In our example, Type and Distribution is what is on the source adapter that we are trying to use as an adapter key). If you look below, this is where the source is getting the key from.
- (in) This represents an contains component. If you wanted to do an equals in this section, you would add “==” or if a specific subnet, you would add “in_net”. Be mindful that if you add the in or in_net, make sure the source field represents the type of lookup that would support this. If you look below at the chart, you will see why we are looking at a contains vs. an equal. You will see that in the CSV, we want to find anything that contains “OS X 10.0” and match it with the source.Type_and_Distribution which has a nomenclature that looks more like “OS X 10.0.7”.
- “device.jamf_adapter.[OS: Type and Distribution])” – Match on the JAMF adapter the previous information to the OS:Type and Distribution . If you were wanting to add this to Qualys, this would look like device.qualys_adapter.[OS: Type and Distribution]
In order to add this formula, please go to the top right corner (settings button). Then the Global Settings Tab and then scroll down to the section that says Custom Enrichment (BETA). Toggle the setting that says “Enable Custom Enrichment” and then enter your statement on the “enrichment statement” field. In the input, this is where you will choose the CSV file that you will upload and upload the file. Please make sure to limit the fields to just what you are uploading and do not include a header. UTF-8 is preferred with this upload.
More Nuance info
Sometimes you simply want to add everything that is on the CSV into the enrichment category. We have a fix for you as well. The way you add EVERYTHING on the CSV without having to adjust the names in the formula after “with” would be to add an “*”. In this example, if we wanted to add everything that was brought in on the CSV above, you would put in the following formula:
enrich 'devices' with (*) on (source.Type_and_Distribution in device.jamf_adapter.[OS: Type and Distribution])
As you can see, everything is mostly the same. However, with the asterisk on the formula, you will find that the field acts as a wildcard. Do not do the wildcard in any other place as the logic of the formula will not allow it.
This is what the field will end up looking like with the wildcard enrichments:
FAQ
- Can it be used on an aggregated field? As mentioned above, no.
- Can it be put on many adapters? You can add the same information to multiple adapters. Once you have added one CSV, click the plus bar below the input and start with your next CSV adapter.
- Is there a limit to the fields? We have not experienced any limit to enrichments however you may want to decide what is the best qty per your use case.
Comments
Awesome - now I need to test this out in our environment
Question - Is the adapter name the same as the adapter connection label or is the the overall adapter name?
Hi Len,
There are instances when you have multiple instances of the same Adapter such as 3 different Active Directory Adapters. To distinguish between the 3 Adapters, you enter an Adapter connection label. Let me know if you have any further questions.
Thank you,
Derek
Thanks - I'm still getting a Wrong query syntax error - I've opened a support case for more help
Len, most often times they aggregated adapter connection name is noted like this: active_directory_adapter . This is a bit wonky as it is not labeled in the UI exactly like how it is written (and we are working to fix that issue). If you export the field that you want with the specific adapter, it is in the aggregated: adapter connections tab. Below is an example of some of them. For the most part, they are pretty straight forward. I will see about adding a bunch to this article to help out. Let me know if you have any issues and we can troubleshoot together.
Christopher,
This is a helpful example. Now how do I do this with a dynamic data source? i.e. update the csv file on a regular basis automatically.
Thanks,
Michael
Michael, currently, we are working on this with the enrichment. You can, however do this completely dynamically with the csv adapter.
Please sign in to leave a comment.