Skip to main content

InQuest Labs Year in Review

Posted on 2020-09-29 by Chris Morrow

Intro

We launched InQuest Labs on August 6th, 2019, as a way to publicly demo some of our R&D and enterprise tech. Unveiled at Black Hat 2019, InQuest Labs made a splash by offering our reputation feed, aggregated IOCs, YARA tools, and a light version of our proprietary Deep File Inspection engine, all for free. While access to InQuest Labs is free, users are limited to 1337 requests per day and cannot download files without requesting an API key. In case you didn't know, we offer an API key for free to anyone interested.

Since launching Labs, we've amassed and hosted a large corpus of indicators. Labs has hosted over 10 million malicious domains, IPs, and URLs and scanned over 1.5 million files with Deep File Inspection. We've seen tremendous growth in usage. We've gone from serving one API key request per week, on average, to an average of 2 per day. In fact, we've generated nearly 100 API keys for Labs users. We're pleased with Labs' growth and will continue to add new features.

Speaking of new features, we just released the official documentation for the InQuest Labs API. The documentation includes detailed explanations of all the endpoints, request code examples in multiple languages, and multiple example responses for various status codes. We hope you find the docs easy to use and useful for integrating InQuest Labs functionality into your environment.

InQuest Labs API Documentation

On the horizon, several improvements are coming to Labs behind the scenes. We're reworking how we internally store data to allow for more complex queries and pivoting around IOCs, overhauling the UI to use more responsive components and updating the styling to Bootstrap 5, and the functionality to upload samples by simply dragging and dropping the file onto the page. We're also updating how saved page states work by using a more compact URL that's backward compatible with the current URL format -- meaning all your bookmarks will still work.

Stats

The following statistics describe the current volume of files, derived indicators, and other artifacts available within the platform.

DFI-DB: Database of files, both malicious and benign, that have gone through Deep File Inspection (DFI) to identify embedded logic, content, and indicators. We've ingested over 1.7M files.

DFI-IOCS: is the database of indicators (IPs, domains, XMP identifiers, etc...), we've pulled over 200M indicators from our corpus of files.

IOC-DB: is a collection of indicators from a variety of company blogs / Tweets, individual researcher Tweets, Github repositories and more. Basically, these are the indicators that folks are talking about, there's over 300k indicators here.

REP-DB: is an aggregation of dozens of public domain reputation feeds, this list is approaching 10M indicators. When we're searching for interesting malware, we'll typically cross-reference the DFI IOCs with the IOC and REP databases. If no one has reported the indicator and no one is talking about the indicator... we may have a new sample worth dissecting deeper.

  • DFIDB: 1,706,147
  • DFIIOC: 208,332,733
  • IOCDB: 312,770
  • REPDB: 9,907,027
  • MIME: 1,696,959
    • DOC: 646,747
    • EML: 26,230
    • OLE: 462,308
    • OTHER: 6,467
    • PPT: 67,019
    • XLS: 497,431

User Feedback

This is the sort of searching and pivoting capability that my colleagues at the Malicious Code team at CERT/CC wished we had access to. It's like someone read our minds and created a product based on our infrastructure wishlist.

—Michael Weber, former Malware Reverse Engineer @ CERT/CC

Requesting API Keys

Our open API is limited to 1337 requests within any 24 hour period. Not to worry though, this measure is solely to prevent abuse. Contact us directly via labs@inquest.net, let us know what you're working on, and we'll give you an unfettered API key (free). Use your key to interact with our API programmatically, or enter it here to uncap your interactive usage.

API Instructions

You can add the API key directly into the UI by clicking the API button to the right of the newsletter.

Enter your key in the presented modal. The key is stored in browser local storage:

You'll know it took if the limits vanished:

Note on the DFI list view, you know have two new visible columns::

The '+' column shows the number of AV engines that detected the file as malicious. The 'lb' column portrays the same sentiment, but where each vendor is given a variable weight. Generally, a total weight above 1 is considered malicious.

If you're looking to access InQuest Labs programmatically, we recommend you take a gander at this Python library / command-line tool: python-inquestlabs

Thank You

The response to InQuest Labs has been quite positive and we've got plenty of roadmap goals for it, so please share any and all feedback. First and foremost, we're working on search performance issues and expect that data migration / cut-over to happen before the end of the year.

Headline image component credit:
https://www.freepik.com/vectors/background
https://www.freepik.com/vectors/birthday
Tags
labs