urlparser.h File Reference

Types for parsing URLs and classifying/handling their host part. More...

#include <array>
#include <cstdint>
#include <iostream>
#include <memory>
#include <optional>
#include <string>
#include <string_view>
#include <unordered_map>
#include <utility>
#include <variant>
#include <vector>

Include dependency graph for urlparser.h:

Go to the source code of this file.

Classes
class	urlparser::host
	hostname, ipv4, or ipv6: whatever a URL's host actually is (RFC 3986's host grammar), wrapped in a uniform API that matches hostname/ipv4/ipv6/url themselves: a constructor that just parses, plus from_url() and str(). More...
class	urlparser::hostname
	A domain name used as a URL's host, e.g. "www.example.com". More...
class	urlparser::ipv4
	An IPv4 address used as a URL's host, e.g. "192.0.2.1". More...
class	urlparser::ipv6
	An IPv6 address used as a URL's host, e.g. "2001:db8::1". More...
class	urlparser::psl
class	urlparser::scp_url
	Parses an SSH "scp-like" address - user@host:path (what scp, rsync, sshfs, and git all accept, e.g. "git@github.com:user/repo.git") - by normalizing it to a standard ssh://user@host/path URI and handing that to url. Not URI syntax per RFC 3986 (that colon is a path separator, not a port), so plain url() either misparses it or throws. More...
class	urlparser::url
	Represents a URL. More...
class	urlparser::version
	Exposes the library version (from the URLPARSER_VERSION_* macros) at runtime, without needing to #include the macros directly. More...

Namespaces
namespace	urlparser
namespace	urlparser::detail
	The library's Public Suffix List matcher.
namespace	urlparser::idna
	IDNA (Internationalized Domain Names in Applications) support.
namespace	urlparser::percent_codec
	Percent-encoding (RFC 3986 "quote") / percent-decoding ("unquote").

Macros
#define	_URLPARSER_VERSION_STR(a, b, c)   _URLPARSER_VERSION_XSTR(a,b,c)
#define	_URLPARSER_VERSION_XSTR(a, b, c)   #a"."#b"."#c
#define	URLPARSER_PUBLIC_SUFFIX_LIST_URL   "https://publicsuffix.org/list/public_suffix_list.dat"
#define	URLPARSER_VERSION    URLPARSER_VERSION_ENCODE(URLPARSER_VERSION_MAJOR,URLPARSER_VERSION_MINOR,URLPARSER_VERSION_PATCH)
#define	URLPARSER_VERSION_ENCODE(maj, min, pat)   (((maj)*10000)+((min)*100)+(pat))
#define	URLPARSER_VERSION_HEX    ((URLPARSER_VERSION_MAJOR << 16) | (URLPARSER_VERSION_MINOR << 8) | URLPARSER_VERSION_PATCH)
#define	URLPARSER_VERSION_MAJOR   2
#define	URLPARSER_VERSION_MINOR   1
#define	URLPARSER_VERSION_PATCH   0
#define	URLPARSER_VERSION_STRING    _URLPARSER_VERSION_STR(URLPARSER_VERSION_MAJOR,URLPARSER_VERSION_MINOR,URLPARSER_VERSION_PATCH)

Typedefs
using	urlparser::git_url = scp_url
	Alias for scp_url. Most callers reaching for this class are parsing a git remote specifically (its most common real-world use, e.g. "git@github.com:user/repo.git") - scp_url itself is named for the underlying address syntax (shared with scp/rsync/sshfs too), not tied to git, so this alias exists purely so the name at a git-focused call site can say what it's for.
using	urlparser::query_params = std::vector< std::string >
	A vector of strings representing query parameters in a URL.

Functions
std::string	urlparser::build_search_params (const std::vector< std::pair< std::string, std::string > > &pairs)
	Build an application/x-www-form-urlencoded query string from ordered key/value pairs - the inverse of url::get_search_params() (url::search_params). Matches WHATWG's own query-serialization rules, verified against ada_url.replace_search_params(): a space becomes '+' (not "%20"), the safe/unescaped set is ASCII alphanumeric plus * - . _ only - narrower than percent_codec::unreserved_set(), which also leaves '~' unescaped; a query string does not. A repeated key (multiple pairs with the same first element) writes multiple "key=value" entries, in the order given - this never deduplicates or reorders.
std::string	urlparser::build_search_params (const url::search_params &params)
	Convenience overload: build from a url::search_params map (url::get_search_params()'s return type) instead of an ordered pair list. Each key's values are written out in their own vector order, but the keys themselves follow std::unordered_map's iteration order - unspecified, and not guaranteed to match the original query string's key order. Use the vector<pair> overload above when order matters.
std::string	urlparser::normalize (std::string_view input)
	Convenience wrapper: url(input).normalized(). Prefer calling .normalized() directly on a url you already have parsed - this just parses one and immediately normalizes it, so it's no faster than doing that yourself, only more convenient for a one-off string-in/string-out call.
std::ostream &	operator<< (std::ostream &os, const urlparser::host &dt)
	Output stream operator for the host variant (hostname/ipv4/ipv6).
std::ostream &	operator<< (std::ostream &os, const urlparser::hostname &dt)
	Output stream operator for hostname.
std::ostream &	operator<< (std::ostream &os, const urlparser::ipv4 &dt)
	Output stream operator for ipv4.
std::ostream &	operator<< (std::ostream &os, const urlparser::ipv6 &dt)
	Output stream operator for ipv6.
std::ostream &	operator<< (std::ostream &os, const urlparser::query_params &dt)
	Output stream operator for query_params.
std::ostream &	operator<< (std::ostream &os, const urlparser::url &dt)
	Output stream operator for url.

Variables
constexpr bool	urlparser::DEFAULT_IGNORE_WWW = false

Detailed Description

Types for parsing URLs and classifying/handling their host part.

RFC 3986 defines a URL's host grammar as:

host = IP-literal / IPv4address / reg-name

i.e. a host is either an IP address or a domain name - never both, never something in between. This library's types mirror that split directly:

• hostname is a domain name ("www.example.com"): subdomain, domain, and suffix, via a lazily-computed Public Suffix List lookup.
• ipv4 / ipv6 are IP addresses: no PSL, no subdomain/domain/suffix (those concepts don't apply to "192.0.2.1"), but arithmetic (++, –, +=, -=), comparisons, and integer conversions instead.
• host = std::variant<hostname, ipv4, ipv6> is whichever of the three a given URL's host actually is - what url::host() returns, and what parse_host()/parse_host_from_url() build directly.

All are plain value types: no PIMPL, no heap allocation for the object itself beyond what a std::string/std::variant already needs. The only deferred work is the PSL-dependent domain/suffix split (hostname) and, for url, building/classifying its host at all - both are computed once, on first access, and cached in mutable members.

Example usage:

urlparser::url u("https://www.example.com/path");
std::cout << "protocol: " << u.protocol() << "\n";
if (auto* h = std::get_if<urlparser::hostname>(&u.host())) {
    std::cout << "domain: " << h->domain() << ", suffix: " << h->suffix() << "\n";
}
 
urlparser::url ipurl("http://192.168.1.1:8080/");
if (auto* v4 = std::get_if<urlparser::ipv4>(&ipurl.host())) {
    std::cout << "IP as int: " << v4->to_uint32() << "\n";
    std::cout << "next IP: " << (*v4 + 1).str() << "\n";
}

Macro Definition Documentation

_URLPARSER_VERSION_STR

#define _URLPARSER_VERSION_STR	(		a,
			b,
			c
	)		_URLPARSER_VERSION_XSTR(a,b,c)

_URLPARSER_VERSION_XSTR

#define _URLPARSER_VERSION_XSTR	(		a,
			b,
			c
	)		#a"."#b"."#c

URLPARSER_PUBLIC_SUFFIX_LIST_URL

#define URLPARSER_PUBLIC_SUFFIX_LIST_URL   "https://publicsuffix.org/list/public_suffix_list.dat"

URLPARSER_VERSION

#define URLPARSER_VERSION    URLPARSER_VERSION_ENCODE(URLPARSER_VERSION_MAJOR,URLPARSER_VERSION_MINOR,URLPARSER_VERSION_PATCH)

URLPARSER_VERSION_ENCODE

#define URLPARSER_VERSION_ENCODE	(		maj,
			min,
			pat
	)		(((maj)*10000)+((min)*100)+(pat))

URLPARSER_VERSION_HEX

#define URLPARSER_VERSION_HEX    ((URLPARSER_VERSION_MAJOR << 16) | (URLPARSER_VERSION_MINOR << 8) | URLPARSER_VERSION_PATCH)

The version of liburlparser in hex: (major << 16) | (minor << 8) | (patch).

URLPARSER_VERSION_MAJOR

#define URLPARSER_VERSION_MAJOR   2

URLPARSER_VERSION_MINOR

#define URLPARSER_VERSION_MINOR   1

URLPARSER_VERSION_PATCH

#define URLPARSER_VERSION_PATCH   0

URLPARSER_VERSION_STRING

#define URLPARSER_VERSION_STRING    _URLPARSER_VERSION_STR(URLPARSER_VERSION_MAJOR,URLPARSER_VERSION_MINOR,URLPARSER_VERSION_PATCH)

The version string of liburlparser, e.g. "2.0.0".

Function Documentation

operator<<() [1/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::host &	dt
	)

Output stream operator for the host variant (hostname/ipv4/ipv6).

operator<<() [2/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::hostname &	dt
	)

Output stream operator for hostname.

operator<<() [3/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::ipv4 &	dt
	)

Output stream operator for ipv4.

operator<<() [4/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::ipv6 &	dt
	)

Output stream operator for ipv6.

operator<<() [5/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::query_params &	dt
	)

Output stream operator for query_params.

operator<<() [6/6]

std::ostream & operator<<	(	std::ostream &	os,
		const urlparser::url &	dt
	)

Output stream operator for url.