Data Parsing Functions

Functions for the parsing of data.

Regular Expressions

re2

The parse::re2 function applies a regex to a string, pulling out matching parts.

parse::re2(pattern, string)

Return types:

• No capture groups: Returns the matched substring.
• Unnamed capture groups: Returns a list of captured substrings in order.
• Named capture groups: Returns a map with group names as keys.

Note: Mixing named and unnamed groups isn't allowed.

If there's no match, an error occurs. Use parse::re2_all to check matches without errors by ensuring the result's length is greater than zero.

Patterns mix literals and special operators. In RedTeamer, escape backslashes as \\.

Special Operators

Zero-width Operators

re2_all

parse::re2_all finds all regex matches in a string.

parse::re2_all(pattern, string)

Outputs depend on capture groups:

• No capture groups: List of matched strings.
• Unnamed capture groups: List of lists for each match.
• Named capture groups: List of maps for each match.

Use parse::re2_all to confirm matches by checking if the result list's length is greater than zero.

parse::re2_all("[a-z]+", "1234abcd5678efgh9")
// returns ["abcd", "efgh"]

length(parse::re2_all("[a-z]+", "1234abcd5678efgh9")) // returns 2
length(parse::re2_all("[a-z]+", "123456789")) // returns false

Number Parsing

int

The parse::int function interprets the provided string as an integer representation in the specified base and returns the resulting number. The base must range between 2 and 62, inclusive.

All bases first utilize Arabic numerals 0 through 9. Bases between 11 and 36 (inclusive) use case-insensitive Latin letters to denote higher unit values. Bases 37 and above use lowercase Latin letters followed by uppercase Latin letters.

parse::int("100", 10)             // returns 100
parse::int("FF", 16)              // returns 255
parse::int("-10", 16)             // returns -16
parse::int("1011111011101111", 2) // returns 48879
parse::int("aA", 62)              // returns 656

If the given string contains non-digit characters or digit characters too large for the specified base, parse::int will trigger an error.

parse::int("12", 2)
// Error: Invalid function argument
// Invalid value for "number" parameter: cannot parse "12" as a base 2 integer.

si

Parses an SI string into its numeric value. Input is a string (si).

// Parses "1K" to its numeric value, 1000
parse::si("1K")

// Parses "5M" to its numeric value, 5000000
parse::si("5M")

// Parses "2G" to its numeric value, 2000000000
parse::si("2G")

// Parses "750m" to its numeric value, 0.75
parse::si("750m")

// Parses "3.5T" to its numeric value, 3500000000000
parse::si("3.5T")

// Parses "100n" to its numeric value, 0.0000001
parse::si("100n")

String Parsing

fields

Splits a string by whitespace into a list of strings. Input is a string (str).

// Splits a simple string into fields
parse::fields("field1 field2 field3")
// Result: ["field1", "field2", "field3"]

// Parses a command output to extract IP addresses
parse::fields("192.168.1.1 192.168.1.2 192.168.1.3")
// Result: ["192.168.1.1", "192.168.1.2", "192.168.1.3"]

// Extracts usernames from a space-separated list
parse::fields("admin user1 user2 pentester")
// Result: ["admin", "user1", "user2", "pentester"]

// Parses a network scan result to list open ports
parse::fields("80/tcp 443/tcp 22/tcp")
// Result: ["80/tcp", "443/tcp", "22/tcp"]

lines

Splits a multiline string into a list of lines. Input is a string (str).

// Splits a simple multiline string into lines
parse::lines("line1\nline2\nline3")
// Result: ["line1", "line2", "line3"]

// Parses a log file to analyze each entry
parse::lines("2024-10-07 10:00:00 Connection from 192.168.1.1\n2024-10-07 10:05:00 Connection from 192.168.1.2")
// Result: ["2024-10-07 10:00:00 Connection from 192.168.1.1", "2024-10-07 10:05:00 Connection from 192.168.1.2"]

// Processes a configuration file to extract settings
parse::lines("setting1=value1\nsetting2=value2\nsetting3=value3")
// Result: ["setting1=value1", "setting2=value2", "setting3=value3"]

// Analyzes the output of a vulnerability scan
parse::lines("Vulnerability: CVE-2024-1234\nSeverity: High\nDescription: Buffer overflow in XYZ service")
// Result: ["Vulnerability: CVE-2024-1234", "Severity: High", "Description: Buffer overflow in XYZ service"]

Network Parsing

domains

Extracts potential domain names from a string into a list. Input is a string (str).

// Extracts domains from a simple sentence
parse::domains("Visit example.com or example.org")
// Result: ["example.com", "example.org"]

// Extracts domains from a network scan report
parse::domains("Discovered domains: internal.example.com, dev.example.org")
// Result: ["internal.example.com", "dev.example.org"]

// Identifies domains from a web scraping result
parse::domains("Links found: shop.example.com, blog.example.net")
// Result: ["shop.example.com", "blog.example.net"]

emails

Extracts email addresses from a string into a list. Input is a string (str).

// Extracts emails from a contact list
parse::emails("Contact us at support@example.com or info@example.com.")
// Result: ["support@example.com", "info@example.com"]

// Parses a data breach dump to find compromised emails
parse::emails("Leaked emails: user1@compromised.com, admin@breached.net")
// Result: ["user1@compromised.com", "admin@breached.net"]

// Extracts emails from a website's contact page
parse::emails("Reach out to sales@company.com or hr@company.com")
// Result: ["sales@company.com", "hr@company.com"]

ipv4_addresses

Extracts IPv4 addresses from a string into a list. Input is a string (str).

// Extracts IPs from a simple list
parse::ipv4_addresses("IPs: 192.168.1.1, 10.0.0.1")
// Result: ["192.168.1.1", "10.0.0.1"]

// Parses a firewall log to identify source IPs
parse::ipv4_addresses("Blocked IPs: 203.0.113.5, 198.51.100.7")
// Result: ["203.0.113.5", "198.51.100.7"]

// Extracts IPs from a vulnerability scan report
parse::ipv4_addresses("Vulnerable hosts: 172.16.0.10, 192.168.100.20")
// Result: ["172.16.0.10", "192.168.100.20"]

// Identifies IPs from a network configuration file
parse::ipv4_addresses("Configured IPs: 10.1.1.1, 10.1.1.2")
// Result: ["10.1.1.1", "10.1.1.2"]

ipv6_addresses

Extracts IPv6 addresses from a string into a list. Input is a string (str).

// Extracts a single IPv6 address
parse::ipv6_addresses("IPv6: 2001:0db8:85a3:0000:0000:8a2e:0370:7334")
// Result: ["2001:0db8:85a3:0000:0000:8a2e:0370:7334"]

// Parses a network log to find IPv6 addresses
parse::ipv6_addresses("Accessed by: 2001:db8::1, 2001:db8::2")
// Result: ["2001:db8::1", "2001:db8::2"]

// Extracts IPv6 addresses from a configuration file
parse::ipv6_addresses("Configured IPv6: fe80::1, fe80::2")
// Result: ["fe80::1", "fe80::2"]

// Identifies IPv6 addresses from a scan result
parse::ipv6_addresses("Detected IPv6 hosts: 2001:db8:abcd:0012::0, 2001:db8:abcd:0012::1")
// Result: ["2001:db8:abcd:0012::0", "2001:db8:abcd:0012::1"]

mac_addresses

Extracts MAC addresses from a string into a list. Input is a string (str).

// Extracts MAC addresses from a simple list
parse::mac_addresses("Device MACs: 00:1A:2B:3C:4D:5E, 01:2B:3C:4D:5E:6F")
// Result: ["00:1A:2B:3C:4D:5E", "01:2B:3C:4D:5E:6F"]

// Parses a network scan to find device MACs
parse::mac_addresses("Detected MACs: 02:42:ac:11:00:02, 02:42:ac:11:00:03")
// Result: ["02:42:ac:11:00:02", "02:42:ac:11:00:03"]

// Extracts MAC addresses from a DHCP log
parse::mac_addresses("Assigned MACs: 00:0C:29:3E:1B:4F, 00:0C:29:3E:1B:50")
// Result: ["00:0C:29:3E:1B:4F", "00:0C:29:3E:1B:50"]

// Identifies MAC addresses from a security audit
parse::mac_addresses("Unauthorized MACs: 00:14:22:01:23:45, 00:14:22:01:23:46")
// Result: ["00:14:22:01:23:45", "00:14:22:01:23:46"]

urls

Extracts URLs from a string into a list. Input is a string (str).

// Extracts URLs from a simple sentence
parse::urls("Check https://example.com and http://example.org")
// Result: ["https://example.com", "http://example.org"]

// Extracts URLs from a web scraping result
parse::urls("Found links: https://shop.example.com, http://blog.example.net")
// Result: ["https://shop.example.com", "http://blog.example.net"]

// Identifies URLs from a vulnerability report
parse::urls("Vulnerable URLs: http://vulnerable.com/login, https://insecure.org")
// Result: ["http://vulnerable.com/login", "https://insecure.org"]

Data Parsing

bytes

Parses a string representing bytes into the number of bytes it represents. Input is a string (size).

// Parses a simple byte size string
parse::bytes("1024B")
// Result: 1024

// Converts kilobytes to bytes
parse::bytes("1KB")
// Result: 1024

// Converts megabytes to bytes
parse::bytes("5MB")
// Result: 5242880

// Parses a log entry to determine data transfer size
parse::bytes("Data transferred: 2GB")
// Result: 2147483648

// Converts gigabytes to bytes for storage analysis
parse::bytes("10GB")
// Result: 10737418240

credit_cards

Extracts credit card numbers from a string into a list. Input is a string (str).

// Extracts credit card numbers from a simple list
parse::credit_cards("Cards: 4111 1111 1111 1111, 5500 0000 0000 0004")
// Result: ["4111 1111 1111 1111", "5500 0000 0000 0004"]

// Parses a data breach dump to find exposed credit card numbers
parse::credit_cards("Leaked cards: 3782 822463 10005, 6011 1111 1111 1117")
// Result: ["3782 822463 10005", "6011 1111 1111 1117"]

// Extracts credit card numbers from a payment log
parse::credit_cards("Processed payments: 5555 5555 5555 4444, 4012 8888 8888 1881")
// Result: ["5555 5555 5555 4444", "4012 8888 8888 1881"]

v4_uuids

Extracts v4 UUIDs from a string into a list. Input is a string (str).

// Extracts a single v4 UUID
parse::v4_uuids("UUIDs: a8098c1a-f86e-11da-bd1a-00112444be1e")
// Result: ["a8098c1a-f86e-11da-bd1a-00112444be1e"]

// Parses a log file to find session UUIDs
parse::v4_uuids("Session IDs: 550e8400-e29b-41d4-a716-446655440000, 123e4567-e89b-12d3-a456-426614174000")
// Result: ["550e8400-e29b-41d4-a716-446655440000", "123e4567-e89b-12d3-a456-426614174000"]

// Extracts UUIDs from a configuration file
parse::v4_uuids("Configured UUIDs: 9b2c4d16-8e4b-4f3b-9f1b-2b2b2b2b2b2b")
// Result: ["9b2c4d16-8e4b-4f3b-9f1b-2b2b2b2b2b2b"]

// Identifies UUIDs from a database dump
parse::v4_uuids("Database entries: 6f1e3b4a-2b2b-4b4b-8e4b-9b2c4d16b2b2")
// Result: ["6f1e3b4a-2b2b-4b4b-8e4b-9b2c4d16b2b2"]