TIDE is Threat Intelligence Data Exchange. It is effectively a very large Threat Database that is managed by the Infoblox Threat Intelligence Team. From its database, the Infoblox RPZ feeds are generated.

In the CSP > Research > Active Indicators, we can't see RPZ membership of threats. This is expected as this page is a simple GUI interface to the Threat Intel database. RPZ membership can be found in Dossier lookups or (ideally) via API lookups.

Infoblox has harmless test domains that are in various RPZ feeds. This allows you to test that a given RPZ feed is active and working as well as generate example logs, etc.

Test RPZ data here.

Get all custom RPZ feeds and Country Based RPZ feeds

curl -s -X GET "https://csp.infoblox.com/api/custom_feeds/v1/feeds" -H "Authorization: Token token=APIKEY_HERE" -H "Content-Type: application/json"

You will get a lot of data back. Each feed has an ID. Use that to limit further queries to pull just that feed.

curl -s -X GET "https://csp.infoblox.com/api/custom_feeds/v1/feeds/<ID>" -H "Authorization: Token token=APIKEY_HERE" -H "Content-Type: application/json"

• IID
• AISCOMM
• EmergingThreats
• FarsightSecurity
• SURBL (no longer available?)
• 0011A00002BcDwEFG

Infoblox Curated Data

• IID:ANALYST
• IID:CRIME
• IID:DTQ_IP
• IID:IID
• IID:IID Internal
• IID:IID_BH_DECLARED
• IID:IID_IRD
• IID:IID_TTL
• IID:LookalikeDomains
• IID:OTHER
• IID:PARTNER
• IID:PORTAL_AK
• IID:POWERSHARK
• IID:RateShark
• IID:IB_NOC
• IID:IB_PM
• IID:IB_DS
• IID:IB_SE

Infoblox 3rd Party Data

• AISCOMM:AIS-COMMERCIAL (Department of Homeland Security's (DHS) Automated Indicator Sharing (AIS))
• EmergingThreats:Hostnames_Feed (ProofPoint)
• EmergingThreats:IP_Feed (ProofPoint)
• EmergingThreats:URL_Feed (ProofPoint)
• FarsightSecurity:NOD (Farsight Security - now part of DomainTools)
• SURBL:FRESH
• SURBL:SURBL_multi_list

SUBRL is SPAM URI (Uniform Resource Identifier) Real-time Block List.

• 0011A00001AaA1aAAA:name-of-custom-data-profile

You can upload data to TIDE via the BloxOne portal (Manage > TIDE Data > Data Upload )

Documentation here.

Example XML file here:

<feed>
 <profile>SampleProfile</profile>
 <record_type>host</record_type>
 <record>
   <host>www.testtesttest0.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest1.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest2.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest3.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest4.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest5.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest6.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest7.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest8.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
 <record>
   <host>www.testtesttest9.com</host>
   <property>Phishing_Phish</property>
   <detected>20230101T154742Z</detected>
 </record>
</feed>

JSON FILE

Notice that the second domain doesn't have a duration.

{ 
 "feed": { 
 "record_type": "host", 
 "record": [  
 {"host": "www.baddomain.com", "property": "Scanner_Generic", "detected": "19980927T154242Z", "duration": "1y0m0w0d42h"}, 
 {"host": "www.example.com", "property": "Phishing_Phish", "detected": "20170602T154742Z"}  ] 
 } 
}

Upload with CURL

 curl -H "AUTHORIZATION:TOKEN APIKEY" -H 'Content-Type: application/json;' -X POST "https://csp.infoblox.com/tide/api/data/batches?profile=SampleDataProfile" --data-binary '@/home/name/DATA_FILE_NAME.json'

The recommended limit for the number of records in a given data submission is 50,000. The maximum number of records should be no more than 60,000 at this point in time.

Documentation here, here, and here.

curl -X GET "https://csp.infoblox.com/tide/api/data/threats?type=host&threat_level_from=80&class=suspicious&field=host,threat_level&period=24h"  -H "Authorization: Token token=PUT_KEY_HERE" > tide-output

cat tide-output | sed s/},{\"host\":\"/\\t/g | sed s/\",\"threat_level\":/\\t/g | sed s/\\t80//g | sed s/\\t100//g | sed s/\\t/\\r\\n/g | sed s/\{\"threat\"\:\\[\{\"host\":\"//g | sed s/\}\],\"record_count\"//g | sed s/\:.*$//g > palo-list

sudo cp ./palo-list /var/www/html/

Make sure NGINX is installed and serving up /var/www/html. Edit as needed. Add certificates as needed.#

On PAN-OS Web UI

• Objects > External Dynamic Lists > Add :
• Type = Domain List (So we can use it in Anti-Spyware Profiles)
• Tick “Automatically expand to include sub-domains”
• Set “Check for updates” as required
• Click “Test Source URL”

You can update the External Dynamic List in Palo Alto Networks to use the following URL and download directly from CSP

https://apikey:API-KEY-HERE@csp.infoblox.com/tide/api/data/threats?data_format=csv&field=domain&rlimit=10&type=host&class=cryptocurrency

Note the use of the username apikey and also an API key.

If you set a Server Certificate in the EDL you can also user username/password (e.g. apiuser and APIKEY) in addition to

csp.infoblox.com/tide/api/data/threats?data_format=csv&field=domain&rlimit=10&type=host&class=cryptocurrency

I got this working for domains but not IP feeds. URL feeds don't work directly from Infoblox CSP because the Palo Alto can't handle the inclusion of

https://

at the start when setting a URL feed.

PROPERTY=Suspicious_Generic
LIMIT=100
MIN_THREAT_LEVEL=80
SOURCE=iid
AGE=7d
APIKEY=APIKEY

curl -X GET -H "Authorization: Token token=$APIKEY" "https://csp.infoblox.com/tide/api/data/threats/state?type=host&profile=$SOURCE&property=$PROPERTY&threat_level_from=$MIN_THREAT_LEVEL&period=$AGE&rlimit=$LIMIT&field=host,detected" | sed s/,/\\n/g | grep host | awk -F "\"" '{print $4}' > suspicious-domains-last-week.txt

Get all bad nameservers from the last year.

PROPERTY=MaliciousNameserver_Generic
OUTPUT="threat-domain-$PROPERTY.txt"
LIMIT=100
MIN_THREAT_LEVEL=80
SOURCE=iid
AGE=1y
APIKEY=SET_APIKEY
curl -X GET -H "Authorization: Token token=$APIKEY" "https://csp.infoblox.com/tide/api/data/threats/state?type=host&profile=$SOURCE&property=$PROPERTY&threat_level_from=$MIN_THREAT_LEVEL&period=$AGE&rlimit=$LIMIT&field=host,detected" | sed s/,/\\n/g | grep host | awk -F "\"" '{print $4}' > $OUTPUT

curl -s -X GET -H "Authorization: Token token=$API_KEY" "https://csp.infoblox.com/tide/api/data/threats/host/hourly?property=Policy_NewlyObservedDomains&data_format=csv&field=host" > new-domains.txt

{{TIDE_url}}/data/threats/host/recent?fields=detected,property,host&data_format=csv

{{TIDE_url}}/data/threats/host/daily?fields=detected,property,host&data_format=csv

/tide/api/data/threats?type=host&class=suspicious&field=host,threat_level,detected,notes&period=24h&rlimit=50000

Using GREP, show top 10 suspicious emergent domains from Infoblox that have been added in the last 48 hours. This list can be fed to External Dynamic List for third party security vendors.

curl -X GET -H "Authorization: Token token=APIKEY" "https://csp.infoblox.com/tide/api/data/threats/state?type=host&profile=iid&property=Suspicious_EmergentDomain&threat_level_from=80&period=48h&rlimit=10&field=host,detected" | sed s/,/\\n/g | grep host | awk -F "\"" '{print $4}' > emergent-domains-last-48h.txt

https://csp.infoblox.com/tide/api/v1/iid_atp/dossier/threat_counts

Show all critical MalwareC2DGA threats detected in the last 48 hours. Limit results to 10 items.

curl -X GET -H "Authorization: Token token=APIKEY" "https://csp.infoblox.com/tide/api/data/threats/state?type=host&profile=iid&class=MalwareC2DGA&threat_level_from=100&rlimit=10&period=48h&field=host,property,threat_level,detected" > critical-malware-threats-last-2days.txt

Main bit

curl -X GET -H "Authorization: Token token=APIKEY" "https://csp.infoblox.com/tide/api/data/threats/state?type=..." > critical-malware-threats-last-2days.txt

… bit - in this case we change class to property = MalwareC2DGA_Locky

host&profile=iid&property=MalwareC2DGA_Locky&threat_level_from=100&rlimit=10&period=48h&field=host,property,threat_level,detected

• Type = Host (type=host)
• Profile = IID (profile=iid)
• Threat Level >= 100 (threat_level_from=100)
• Max Results = 10 (rlimit=10)
• Period = 24h (Results added to TIDE in last 24 Hours)
• Field = detected,host,property (Data to include in result)
• Property = MalwareC2DGA_Locky (Property to detect)
• Class = MalwareC2DGA (Class to detect)

Format is the following. Replace type=… with examples below

curl -X GET 'https://csp.infoblox.com/tide/api/data/threats?type=...'  -H 'Authorization: Token token=<your token>'

Retrieve notes in TIDE for a domain example.com:

type=host&profile=IID&host=example.com

Retrieve new suspicious indicators:

type=host&profile=IID&class=suspicious&rlimit=2

(Possibly) “Absolutely safe to block” and almost certainly not available via any other source or vendor today as verified. However, this is only true of the results where the threat_level >= 50.

type=host&class=suspicious&field=host,detected,threat_level,notes&period=24h

/tide/api/services/intel/lookup/indicator/host?value=example.com&wait=true&data_format=json

Dossier sources.

• iSIGHT is available as a separate subscription and is not automatically included with Dossier.

Remember, if you only have Business Cloud and a threat exists in one of the “Advanced only” RPZ feeds, you won't see those RPZ feeds be identified if you send a Dossier query for the domain.

In the event that you query Dossier for a domain that is in both AntiMalware and (e.g.) Suspicious-Lookalikes, then, as a Business Cloud customer, you will only see AntiMalware in the “Matched RPZ” list. You will, however, see the relevant “suspicious” tag in the activity timeline which gives you a clue it might also be in suspicious. Only Advanced customers will see (in Dossier) that the domain is in both AntiMalware and Suspicious Lookalikes RPZ feeds.

Simply query for

• type = HOST
• profile = IID Data
• class= MalwareC2
• property = MalwareC2_BackdoorRAT
• threatlevel = 80+
• period = (last) 24h

curl -X GET -H "Authorization: Token token=APIKEY" "https://csp.infoblox.com/tide/api/data/threats?type=host&profile=iid&class=malwarec2&property=malwarec2_backdoorrat&threat_level_from=100&period=24h"

curl -s -X POST "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true" -H "Authorization: Token token=APIKEY" -H "Content-Type: application/json" -d '{"target": {"one": {"type": "host", "target": "google.com","sources":["rpz_feeds"]}}}'

Same command but use GREP to filter out everything but the feed names.

curl -s -X POST "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true" -H "Authorization: Token token=APIKEY" -H "Content-Type: application/json" -d '{"target": {"one": {"type": "host", "target": "google.com","sources":["rpz_feeds"]}}}' | grep feed_name | grep -v ib- | awk -F ":" '{print $2}' | sed 's/\ //g' | sed 's/,//g' | sed 's/"//g'

Sam command but with BASH

#!/bin/bash
DOMAIN=$1
APIKEY="PUT_KEY_HERE"
COMMAND="{\"target\": {\"one\": {\"type\": \"host\", \"target\": \"$1\",\"sources\":[\"rpz_feeds\"]}}}"
echo "List of RPZ feeds that contain $1"
curl -s -X POST "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true" -H "Authorization: Token token=$APIKEY" -H "Content-Type: application/json" -d "$COMMAND" | grep feed_name | grep -v ib- | awk -F ":" '{print $2}' |  sed 's/,//g' | sed 's/"//g'

#!/bin/bash
DOMAIN=$1
APIKEY="PUT_KEY_HERE"
COMMAND="{\"target\": {\"one\": {\"type\": \"host\", \"target\": \"$DOMAIN\",\"sources\":[\"infoblox_web_cat\"]}}}"
echo "Infoblox Web Category for $DOMAIN: "
curl -s -X POST "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true" -H "Authorization: Token token=$APIKEY" -H "Content-Type: application/json" -d "$COMMAND" | grep name | awk -F ":" '{print $2}'

"acs","activity","atp","ccb","custom_lists","dns","gcs","geo","gsb","infoblox_web_cat","inforank","isi
ght","malware_analysis","malware_analysis_v3","pdns","ptr","rlabs","rpz_feeds","rwhois","whitelist","whois","s
sl_cert","urlhaus","nameserver","threatfox"

batch_id
class
confidence
confidence_score
confidence_score_rating
confidence_score_vector
detected
dga
domain
expiration
extended  (can contain {"notes":"notes put here"})
full_profile
host
id
imported
profile
property
received
risk_score
risk_score_rating
risk_score_vector
threat_level
threat_score
threat_score_rating
threat_score_vector
tld
type
up

#!/usr/bin/env python3
# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4

import requests
import json

# Needed to read CLI parameters
import sys

# Needed to check the valid domain name regex
import re

# Needed to manipulate the data
from datetime import datetime


APIKEY = "API_KEY"


def is_valid_domain_name(domain_name):
    # Regular expression pattern to match a valid domain name
    pattern = r"^(?!:\/\/)(?!www\.)(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?\.)+(?:[a-zA-Z]{2,})$"

    # Check if the domain name matches the pattern
    if re.match(pattern, domain_name):
        return True
    else:
        return False

def is_date_in_future(date_str):
    current_date = datetime.now().date()
    date = datetime.strptime(date_str, '%Y-%m-%d').date()

    if date > current_date:
        return True
    else:
        return False

if len(sys.argv) < 1:
    print("Usage: python script_name.py domain-to-test.corp.")
    sys.exit(1)  # Exit the script with a non-zero status code indicating an error

parameter1 = sys.argv[1]  # The first parameter

if not is_valid_domain_name(parameter1):
    print("Error: Parameter ",parameter1," is not a valid domain name.")
    sys.exit(1)


DATA = "{\"target\": {\"one\": {\"type\": \"host\", \"target\": \"" + parameter1 + "\",\"sources\":[\"rpz_feeds\"]}}}"

# Make the cURL request
url = "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Token " + APIKEY
}
response = requests.post(url, data=DATA, headers=headers)

# Check if the request was successful (status code 200)
if response.status_code == 200:
    data = response.json()
    results = data["results"]
    for result in results:
        records = result['data']['records']
        if len(records) == 0:
            print("RPZ for ",parameter1,":NONE")
        else:
            print("RPZ for ",parameter1,":")
            for record in records:
                feed_name = record["feed_name"]
                feed_name = feed_name.ljust(27)
                class_name = record["class"]
                properties = record["property"]
                detected = record["detected"]
                expiration = record["expiration"]
                detected = datetime.strptime(detected, "%Y-%m-%dT%H:%M:%SZ")
                detected = detected.strftime("%Y-%m-%d")
                expiration = datetime.strptime(expiration, "%Y-%m-%dT%H:%M:%SZ")
                expiration = expiration.strftime("%Y-%m-%d")
                result = is_date_in_future(expiration)
                if result is True:
                    active = "Active"
                else:
                    active = "Expired"

                print(feed_name,"\t(",properties,")\t(",active,")",detected,"to",expiration)

else:
    print("Error:", response.status_code)

DATA = "{\"target\": {\"one\": {\"type\": \"host\", \"target\": \"" + parameter1 + "\",\"sources\":[\"infoblox_web_cat\"]}}}"

# Make the cURL request
url = "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Token " + APIKEY
}
response = requests.post(url, data=DATA, headers=headers)



# Check if the request was successful (status code 200)
if response.status_code == 200:
    data = response.json()
    results = data["results"]
    print("")
    for result in results:
        records = result['data']['results']
        if len(records) == 0:
            print("WebCat for ",parameter1,":NONE")
        else:
            for record in records:
                cat_name = record["name"]
                print("WebCat for ",parameter1,":",cat_name)

else:
    print("Error:", response.status_code)


DATA = "{\"target\": {\"one\": {\"type\": \"host\", \"target\": \"" + parameter1 + "\",\"sources\":[\"atp\"]}}}"

# Make the cURL request
url = "https://csp.infoblox.com/tide/api/services/intel/lookup/jobs?wait=true"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Token " + APIKEY
}
response = requests.post(url, data=DATA, headers=headers)



# Check if the request was successful (status code 200)
if response.status_code == 200:
    data = response.json()
    results = data['results'][0]['data']['threat']
    print("")
    if len(results) == 0:
        print("No Threat Data for ",parameter1)
    else:
        print("Threat Data for ",parameter1)
        print("FullProfile\tProperty\t\t\tExpiration\tConfidence\tConfidenceScore\t\tThreatLevel\tThreatScore\tRiskScore")

        for result in results:
            # Keep this IF statement if you only want to inlucde Infoblox data and not SURBL or FARSIGHT
            var_profile = "NA"
            var_confidence = "NA"
            var_expiration = "NA"
            var_full_profile = "NA"
            var_property = "NA"
            var_threat_level = "NA"
            var_confidence_score_rating = "NA"
            var_risk_score_rating = "NA"
            var_threat_score_rating = "NA"





            if 'profile' in result:
                var_profile = result['profile']
            if 'confidence' in result:
                var_confidence = result['confidence']
            if 'expiration' in result:
                var_expiration = result['expiration']
                var_expiration = datetime.strptime(var_expiration, "%Y-%m-%dT%H:%M:%S.%fZ")
                var_expiration = var_expiration.strftime("%Y-%m-%d")
                dateres = is_date_in_future(var_expiration)
                if dateres is False:
                    var_expiration = "Expired"
            if 'full_profile' in result:
                var_full_profile = result['full_profile']
            if 'property' in result:
                var_property = result['property']
            if len(var_property) < 22:
                var_property += "\t"
            if 'threat_level' in result:
                var_threat_level = result['threat_level']
            if 'confidence_score_rating' in result:
                var_confidence_score_rating = result['confidence_score_rating']
            if 'risk_score_rating' in result:
                var_risk_score_rating = result['risk_score_rating']
            if 'threat_score_rating' in result:
                var_threat_score_rating = result['threat_score_rating']


            if result['profile'] == 'IID':
                if var_expiration != 'Expired':
                    print(var_full_profile,"\t",var_property,"\t",var_expiration,"\t",var_confidence,"\t\t",var_confidence_score_rating,"\t\t\t",var_threat_level,"\t\t",var_threat_score_rating,"\t\t",var_risk_score_rating)


else:
    print("Error:", response.status_code)