WhoisParsers

WhoisParsers: Whois and RWhois Parsers and Crawlers C# Library

This library provides two main features:

1. Parsers to read Whois records from offline bulk whois database dumps of IANA organizations (ARIN, AFRINIC, APNIC, LACNIC, and RIPE)

2. Crawlers to retrieve online RWhois data from ARIN Referral Whois servers. This is a partial implementation of RFC 2167 that supports both bulk crawls using the -xfer command and incremental crawls.

This library does NOT provide features to contact the REST APIs such as ARIN's Whois-RWS.

Nuget Package Installation

Create a new console C# project, then install the WhoisParsers NuGet package by using the Visual Studio GUI or by using this command in the Package Manager Console:

Install-Package WhoisParsers

Parsing offline bulk Whois databases

Creating a parser instance for reading ARIN, APNIC, LACNIC, or RIPE databases

var parser = new WhoisParser(new SectionTokenizer(), new SectionParser());

Creating a parser instance for reading AFRINIC databases

var parser = new WhoisParser(new AfrinicSectionTokenizer(), new SectionParser());

Parsing Whois sections

You can get the sample arin.sample.txt file from here.

var parser = new WhoisParser(new SectionTokenizer(), new SectionParser());var sections = parser.RetrieveSections(@"arin.sample.txt");foreach (var section in sections){Console.WriteLine(string.Format(CultureInfo.InvariantCulture, "Section ID: {0}", section.Id));Console.WriteLine(string.Format(CultureInfo.InvariantCulture, "Number of records: {0}", section.Records.Count));Console.WriteLine("---- Section Records:");Console.WriteLine(section);Console.WriteLine();}

Public functions provided by WhoisParser include:

Function	Description
ColumnsPerType	Retrieve a list of unique record names for each type of records in a database dump. Signature variations: (string filePath) - read from text file path (StreamReader reader) - read from an existing reader
RetrieveSections	Retrieve parsed sections from the bulk database. Signature variations: (string filePath) - read from text file path (string filePath, string desiredType) - read sections with only a certain type from text file path (string filePath, HashSet desiredTypes) - read sections with only certain types from text file path (StreamReader reader, string desiredType) - read sections with only a certain type from an existing reader (StreamReader reader, HashSet desiredTypes) - read sections with only certain types from an existing reader
RetrieveSectionsFromString	Retrieve parsed sections from the bulk database where the database is passed in as a string. Signature variations: (string text) - read all sections from the string (string text, string desiredType) - read sections with only a certain type from the string (string text, HashSet desiredTypes) - read sections with only certain types from the string

Incrementing IPv4 and IPv6 IP addresses

The library contains functions to increment IPV4 and (more importantly) IPv6 IP addresses.

using Microsoft.Geolocation.Whois.Utils;...var ipv4Address = IPAddress.Parse("192.168.0.1");Console.WriteLine(string.Format(CultureInfo.InvariantCulture, "Before: {0}, After: {1}", ipv4Address, ipv4Address.Increment()));var ipv6Address = IPAddress.Parse("2001:db8:a0b:12f0::1");Console.WriteLine(string.Format(CultureInfo.InvariantCulture, "Before: {0}, After: {1}", ipv6Address, ipv6Address.Increment()));

The output looks like this:

Before: 192.168.0.1, After: 192.168.0.2
Before: 2001:db8:a0b:12f0::1, After: 2001:db8:a0b:12f0::2

Converting parsed Whois database sections to structured C# objects

Documentation TODO

Crawling an RWhois Referral server

Documentation TODO

Crawling multiple RWhois Referral servers at the same time

Documentation TODO