Extract AWS Inspector data from CLI
In this post I’m going to show you a quick and easy way to use the AWS command line to filter and download data from Inspector. Jumping near the conclusions, here’s a one-liner that gives you a csv-like output filtering on certain fields:
(echo "Severity,Title,Status,Instance,Type,Exploit,Fix,Age" && aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))' | jq -r '.[]| join(",")')
Table of Contents
Summary
Like many other companies, the place I work is fully in the cloud, specifically on AWS. We try to stick with native services instead of relying on third-party tools, and for vulnerability scanning, we use AWS Inspector. This services pretty easy to use and you can easily filter data right from the console, however it doesn’t easily let you download the findings that you might want to process locally. There is an Export Findings button in the GUI but it forces you to place the file into an s3 bucket, and you cannot simply download it directly from your browser. Idk why they thought this is a good idea, but in companies where privileges are actually well managed, this can be an hassle and might force you to set up a dedicated bucket just to get a csv with 5 lines. For example I might have access to Inspector as i'm from the security team, but i cannot create an s3 bucket.
This is how the list of findings looks like from the GUI, note the 2 filters that are applied and the export button:

Context
At work one problem we recently were trying to tackle is how to reduce the amount of vulnerabilities across our AWS estate, especially ones that could be out of SLAs -where SLAs are defined both internally and by compliance regulations. As any other sane workplace we decided that this must be done automatically and we ultimately set up auto-patching on most of our machines using dnf. That's super cool actually, but sadly not the point of this post.
This is working great so far and we were able to basically eliminate any vulnerability out of SLAs as we now apply patches more frequently than the shortest SLA.
Operationally wise, to confirm the results we checked on Inspector console (GUI) and noticed immediately that pretty much all the vulnerabilities were gone and the ones remaining shows a very recent “Age” (number of days from the date of last detection and today). Basically if Age < SLA, we are good.
Now, from a security standpoint the problem is solved* and as it’s a set-and-forget automation that will live on its own from now on; however as any enterprise company, certain area of the business love require reporting. So the next step was to automate the reporting and here we come to the original issue: easily extract data from Inspector.
*there will be exceptions when you want to emergency patch something, but from a day-to-day basis this set up is good enough.
AWS CLI
Intro
We immediately discarded the idea to save the data into an s3 bucket as that’s overkill. Might work for bigger companies with thousands of results from Inspector but i don’t honestly really see the point in overly complicate a simple action that is to download a csv with results. Also, as we set up auto patching, we don’t even expect this csv to have lot of rows in the first place, especially with the right filters.
Before jumping in the cli magic below, official references for Inspector are here:
- https://docs.aws.amazon.com/cli/latest/reference/inspector/
- https://docs.aws.amazon.com/cli/latest/reference/inspector/list-findings.html
Getting our way via CLI and extract data
Now we’re gonna use AWS CLI to download data from Inspector and filter them the way we need. At last we’ll have a csv that mimics what we can see in the table in GUI posted above. Note this will be done with one-liner bash commands you can copy-paste in the terminal but you might want to build a script instead.
Start
Once you log in AWS from your terminal -you only need the readonly role- and set the correct region, this would be the base command to get data from inspector:
aws inspector list-findings
For testing purposes i’ll add a --max-items 1 or 2:
aws inspector list-findings --max-items 1
Now this isn’t much useful as it display the arn of the finding but not the details of the finding itself. For that we need to take the arn returned in the command above use describe-findings like this:
aws inspector describe-findings --finding-arns --finding-arns "arn:aws:inspector:eu-west-1:XXXXXXXX:target/XXXXXXXX/template/XXXXXXXX/run/XXXXXXXX/finding/XXXXXXXX"
This will return a huge json with all the info on the findings, but i’m only interested in a few fields:
severity,title,status,resources[*].id,type,exploitAvailable,fixAvailable,lastObserved
Filters
In order to filter for those, or any other fields you need to add the --query parameter. Note i’ll use the json output as it’s the easier to process for later:
aws inspector2 list-findings --query 'findings[*].{Severity:severity,Title:title,Status:status,Instance:resources[*].id,Type:type,Exploit:exploitAvailable,Fix:fixAvailable,LastObserved:lastObservedAt}' --output json --max-items 1
This will return you a json like this one:
[
{
"Severity": "MEDIUM",
"Title": "CVE-9999-99999 - linux-image-generic",
"Status": "CLOSED",
"Instance": [
"i-XXXXXXXX"
],
"Type": "PACKAGE_VULNERABILITY",
"Exploit": "NO",
"Fix": "YES",
"LastObserved": "2099-99-99T00:00:00.000000+00:00"
}
]
Before going further, i want to add a filter on certain fields. Noticed the status of the evidence above is CLOSED which means that i don’t care about the issue as it’s already solved. The other one i want to use is Exploit, as if the vulnerability is not exploitable i don’t care either. We can use the --filter parameter to apply some logic here:
aws inspector2 list-findings --query 'findings[*].{Severity:severity,Title:title,Status:status,Instance:resources[*].id,Type:type,Exploit:exploitAvailable,Fix:fixAvailable,LastObserved:lastObservedAt}' --output json --max-items 1 --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}'
Post result processing
Now we can process the json and there are 2 things i want to change. First get rid of the array in Instance to flatten the json structure and we can do with some bash magic:
aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json --max-items 1
The output will be a flat json:
[
{
"Severity": "MEDIUM",
"Title": "CVE-9999-99999 - linux-image-generic",
"Status": "CLOSED",
"Instance": "i-XXXXXXXX",
"Type": "PACKAGE_VULNERABILITY",
"Exploit": "NO",
"Fix": "YES",
"LastObserved": "2099-99-99T00:00:00.000000+00:00"
}
]
Then i want to change the LastObserved to be the Age we can see in the Inspector console and for that, we need to add some complexity in the command:
aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json --max-items 2 | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: ((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)} | del(.LastObserved))'
The result is that the line "LastObserved": "2099-99-99T00:00:00.000000+00:00" will be replaced by "Age": 3. Now it seems AWS Inspector count the same day of the discovery as 1 and not 0, so to match the same value you see in the GUI, we need to add 1. We could insert a joke about array starting at 0 but I guess here it make sense as to not confuse this with a zero day.
Anyway, the resulting command is:
aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json --max-items 2 | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))'
Now, if you need to see the data on the terminal you might end here, or maybe replace the --output json with a --output table or similar. My end goal is to create a CSV and that’s why we flatten the json earlier. I use a very straightforward method to do that, with jq:
aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json --max-items 2 | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))' | jq -r '.[]| join(",")'
The output is nice, but is missing the headers:
HIGH,CVE-9999-99999 - linux-image-generic,ACTIVE,i-XXXXXXXX,PACKAGE_VULNERABILITY,YES,YES,4
HIGH,CVE-9999-99999 - linux-image-generic,ACTIVE,i-XXXXXXXX,PACKAGE_VULNERABILITY,YES,YES,4
So again going for the easiest route possible, i just decided to add the header with an echo:
(echo "Severity,Title,Status,Instance,Type,Exploit,Fix,Age" && aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json --max-items 2 | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))' | jq -r '.[]| join(",")')
Severity,Title,Status,Instance,Type,Exploit,Fix,Age
HIGH,CVE-9999-99999 - linux-image-generic,ACTIVE,i-XXXXXXXX,PACKAGE_VULNERABILITY,YES,YES,4
HIGH,CVE-9999-99999 - linux-image-generic,ACTIVE,i-XXXXXXXX,PACKAGE_VULNERABILITY,YES,YES,4
Final Command
You can now save to csv by adding a simple > results.csv at the end of the command. Here the final version for easy copy-paste, removing the limits on the results (notice the previous commands have --max-items X) so to get the full output:
(echo "Severity,Title,Status,Instance,Type,Exploit,Fix,Age" && aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))' | jq -r '.[]| join(",")') > inspector.csv
Reporting
Now we have a csv with all the details inside that we can process further depending on what we need to do. One quick thing i i need is to simply check if there is any number of vulnerability that is above SLA. Let’s say the shortest SLA is 30 days (typical from compliance regulation), there are several method to do so but we can simply add another command to count if there’s any row with Age >= 30 in the csv. While we are here let’s also use a variable to set the filename for the csv and include today’s date:
today_date=$(date +%Y%m%d); filename="inspector_${today_date}.csv"; ((echo "Severity,Title,Status,Instance,Type,Exploit,Fix,Age" && aws inspector2 list-findings --query 'findings[*].{Severity:severity, Title:title, Status:status, Instance:join(``, resources[*].id), Type:type, Exploit:exploitAvailable, Fix:fixAvailable, LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved))' | jq -r '.[]| join(",")') > $filename) && awk -F, 'NR > 1 && $8 >= 30 {count++} END {if (count > 0) print count; else print 0}' $filename
This will both create the csv named like inspector_YYYYMMDD.csv, then read it and print the number of vulns with Age >= 30, which if you set auto patching well enough, should always be 0*.
*Unless there are exploitable vulnerabilities without a fix (for more than 30 days) but you can decide to exclude those as mentioned above
At this point it really depends on what kind of report you need to do. You can alter the logic above to first check the age, and write a CSV only if there is at least one vulnerability older than your SLAs. For that you can do something like this to only check the Age and if any row is >=30, you’ll run one of the command above (here you might wanna start writing a script instead of use one-liners commands):
aws inspector2 list-findings --query 'findings[*].{LastObserved:lastObservedAt}' --filter '{"findingStatus":[{"comparison":"EQUALS","value":"ACTIVE"}],"exploitAvailable":[{"comparison":"EQUALS","value":"YES"}]}' --output json | jq --arg today "$(date +%Y-%m-%d)" 'map(. + {Age: (((($today | strptime("%Y-%m-%d") | mktime) - (.LastObserved | split("T")[0] | strptime("%Y-%m-%d") | mktime)) / 86400 | floor)+1)} | del(.LastObserved)) | map(select(.Age >= 30)) | length'
- ← Previous
Set up this blog with Eleventy - Next →
API Stream Error with Burp
