README

Some core PHP classes, originally created for website and RSS metadata parsing.

Quick Reference

CountryCode - ISO3166 codes and associated data
CountryFlag - flags of all sizes for ISO3166 countries
Cryptor - symmetric encryption
DataStore - abstract SQL storage (base class)
HttpResponse - official response strings
MIMEType - handle ContentTypes
PageParser - parse an HTML page to get metadata
QRCode - create QR Codes
RateLimiter - throttle for multi-transaction processes
SiteIcon - retrieve (and optionally store) site icons ('favicons')
URL - interface to CURL, mostly
UUID - generate unique IDs
Utilities - other utilities

Installing the Utilities library

This library is best installed using composer. If you haven't yet installed composer, get it HERE. To install this library, require it using composer:

composer require guardianproject/core_utilities

Configuring your installation

Before using this library, please familiarize yourself with the configuration file UtilitiesConfig.php, editing as appropriate for your usage, with special attention to where you wish to store your encryption key.

Initializing the Utilities Library

Some of the PHP objects in this library store and retrieve information in small SQLite3 databases. These databases need to be initialized from provided flat-files. The process of initialization is handled by the init.php script. In addition, an encryption key for Cryptor will be created - with configured name in the configured directory - if one does not already exist. In addition, directories used by log files, data files and databases will be created if they do not already exist.

Run this script before using this library.

php init.php

Testing Your Installation

This script tests most provided functions (DataStore, Utilities excepted).

php test/lib_test.php

Using this Library

To use classes from this library in your own software, you only need to set a default timezone (for the classes extending DataStore) and require the library's include file

date_default_timezone_set('UTC');
require_once './inc.php';

Use PHP's use statement to make a class available:

use guardianproject\core_utilities\CountryCode as CountryCode;
try {
	$cc = new CountryCode();
	print "The country name for ISO3166 code [FR] is" . $cc->getCountryName('FR') . "\n";
} catch (Exception $e) {
	print "Error: " . $e->getMessage() . "\n";
}

API Description

CountryCode

Retrieve a country's ISO3166 2-character code using its ISO3166 numeric code, ISO3166 3-character code, or ISO3166 English Language Name. Then retrieve data using the ISO3166 2-character code. Data set includes:

ISO3166 numeric code
ISO3166 3-character code
ISO3166 English-language Name
Internet Top Level Domain
Capital city name (if it has one)
UN Membership
Independent/Territory status
ITU standard dialing codes

Methods with names starting with get or retrieve are synonymous.

Usage

try {
	$cc = new CountryCode($force); // force=true rebuilds database table from provided CSV

	$ccode->countryCodeExists($cc);
	$ccode->countryNameExists($name);
	
	// retrieve useful lists
	
	$ccode->allCountryCodes();
	$ccode->allAlpha3Codes();
	$ccode->allNumericCodes();
	$ccode->allTerritories();
	$ccode->allIndependentNations();
	
	// retrieve full data set using any ISO3166 code
	
	$ccode->getCountryData($cc, $asJSON = null);
	$ccode->retrieveCountryData($cc, $asJSON = null);
	$ccode->getCountryDataUsingAlpha3Code($alpha3) { 
	$ccode->retrieveCountryDataUsingAlpha3Code($alpha3);
	$ccode->getCountryDataUsingNumericCode($num); 
	$ccode->retrieveCountryDataUsingNumericCode($num);

	// if all you have is the country name...
	
	$ccode->getCountryCode($name); 
	$ccode->retrieveCountryCode($name);
	$ccode->retrieveCountryCodeFuzzy($name);
	$ccode->retrieveCountryNamesLike($str);

	// retrieve individual data items via the ISO3166 2-character code
	
	$ccode->getCountryName($cc);
	$ccode->retrieveCountryName($cc);
	$ccode->getNumericCode($cc);
	$ccode->retrieveNumericCode($cc);
	$ccode->getAlpha3Code($cc);
	$ccode->retrieveAlpha3Code($cc);
	$ccode->getCountryCapital($cc);
	$ccode->retrieveCountryCapital($cc);
	$ccode->getInternetTLDForCountryCode($cc);
	$ccode->retrieveInternetTLDForCountryCode($cc);
	$ccode->callingCodesForCountryCode($cc);
	$ccode->ITUCodesForCountryCode($cc);
	$ccode->isUNMember($cc);
	$ccode->isIndependent($cc);
	$ccode->hasCapital($cc);
	$ccode->territoryOf($cc);

	// database maintenance
	$ccode->rebuild();
	$ccode->dump($to_file = null);
}

CountryFlag

Retrieve national flag images (PNG) for all ISO3166 countries, the United Nations (UN) and Antarctica (AQ). Icon-sized files are cached locally; other sizes are retrieved when requested from a CDN. All flag data returned as base64-encoded. Allowable size are as follows:

icon - 40px width
mini - 160px width
normal - 640px width
big - 1280px width
ultra - 2560 width

NOTE: Under some circumstances, emoji flags might have more utility than the icon-sized images. Use method getEmojiForCountryCode() to acquire that proper character.

Usage

try {
	$cf = new CountryFlag($force);	// force=true rebuilds database table from provided CSV

	$cc->getFlagForCountryCode($code, $size = 'icon');
	$cc->getFlagForCountryName($name, $size = 'icon');
	$cc->getEmojiForCountryCode($code);
	$cc->updateFlagIcon($cc);
	$cc->rebuild();					// rebuild icon cache from CDN source
	$cc->dump($to_file = null);
}

Cryptor

Symmetric encryption functions. This work uses PHP's OpenSSL implementation.

Usage

try {
	$cryptor = new Cryptor($bool); // true/false chunked mode

	// before executing any action methods, create an encryption key, to be stored
	// in the location provided in UtilitiesConfig.php
    public static function generateKey($seed_string = null) {

	// use a provided initialization vector instead of an auto-generated one
	$cryptor->setInitializationVector($iv);
	$iv = $cryptor->getInitializationVector();

	// chunked mode
	$cryptor = setChunkMode($boolean);
	
	// encrypt/decrypted provided data
	$blob = $cryptor->encrypt($data);
	$data = $cryptor->decrypt($blob);
	
	// same, but with file artifacts
	$cryptor->encryptFile($fn);
	$cryptor->encryptToFile($fn, $data);
	$cryptor->decryptFile($fn);
	$cryptor->decryptToFile($fn, $blob);
}

MIMEType

Translate between MIME type identifiers (the MIME strings and the associated file extension).

Usage

try {
	$mime = new MIMEType();

	$str = $mime->getFileExtensionForType($mime);
	$str = $mime->getTypeforFileExtension($fext);
	$str = $mime->getApplicationForType($mime);
	$bool = $mime->isSimpleContentType($simple_type, $fext);
}

UUID

Unique identifiers are handy. The UUID class synthesizes the several standardized types if you've got specific needs, or does the lifting if you dont.

Usage

$namespace = '1546058f-5a25-4334-85ae-e68f2a44bbaf';
$str = sha1(mt_rand(0,100000) . "PHP");

// using provided namespace and randomization string

$v3uuid = UUID::v3($namespace, $str);
print "v3: " . $v3uuid . "\n";
$v5uuid = UUID::v5($namespace, $str);
print "v5: " . $v5uuid . "\n";

// using provided namespace

$v3uuid = UUID::v3($namespace);
print "v3: " . $v3uuid . "\n";
$v5uuid = UUID::v5($namespace);
print "v5: " . $v5uuid . "\n";

// using defaults 

$v3uuid = UUID::v3();
print "v3: " . $v3uuid . "\n";
$v5uuid = UUID::v5();
print "v5: " . $v5uuid . "\n";

$v4uuid = UUID::v4();
print "v4: " . $v4uuid . "\n";

print "PHP uniqid: " . uniqid() . "\n";

$ruuid = UUID::next();
print " r: " . $ruuid . "\n";

HttpResponse

It's sometimes helpful to know the official HTTP response message text, especially when logging or reporting an error. Both long, and short, response strings are available.

Usage

try {
	$hr = new HttpResponse($force);
	
	if ($hr->httpCodeExists(500)) {
			print "500: " . json_encode($hr->retrieveDescriptions(500)) . "\n";
	}

	// Need to find out if a URL exists and/or has content? Get the descriptive response
	// strings for the URL provided
	$descriptions = $hr->urlResponse($url);
}

URL

Wrapper around PHP's CURL object.

Usage

try {
	$url = new URL();

	// optional on ping(), get(), acquire(); ineffective with post(), put()	
	$uh->setUrl($url);
	
	// useful setters
	$uh->setRequestHeader($name, $content);
	$uh->setCredentials($user, $pass);
	$uh->setTimeout($secs);
	
	$bool = $uh->isLikelyAUrl($string);
	$bool = $uh->lastRequestTimedOut();
	
	// page metadata returned
	$info = $uh->ping($url, $follow = false, $with_hdrs = false);	
	print_r($info);
	
	// 'body' element contains result, other elements are informational
	$res = $uh->get($url, $follow = false, $with_hdrs = false);
	$res = $uh->acquire($url, $follow = false, $with_hdrs = false);

	$res = $uh->post($url, $params, $follow = false);
	$res = $uh->put($url, $params, $follow = false);
	print_r($res);
}

QRCode

Generate QRCode images. See documentation HERE.

Usage

try {
	$qr = new QRCode();

	// optional setters
	$qr->setCustomProtocol($p);			// to use an app-dependent scheme like 'foo://'
	$qr->setFile($name);				// name of output file
	$qr->setDirectory($directory);		// directory into which output file is loaded
	$qr->setSizeInPixels($size);
	$qr->setFormat($format);			// file type
	$qr->setECC($ecc);
	$qr->setCharset($charset);	
	
	$qr->setData('https://guardianproject.info'); // if custom protocol, just data, no scheme
	$qr->generateQRCode();
}

SiteIcon

Check, acquire and store site icons ('favicons'). Index these using the site's homepage URL. Map provided URLs to the proper icon.

Usage

try {
	$si = new SiteIcon();
	
	$si->setVerboseMode($bool);

	// get icon for specified site URL, either from local storage (if available) or
	// direct from site.  Store result if found remotely.
	$si->acquireIcon($url, $direct = null);

	// for a given URL, find the website's icon URL
	$si->findIconUrl($url);
		
	// is there a locally-stored icon for this site URL?  If so retrieve it
	if ($si->iconExists($url)) {
		$si->retrieveIcon($url);
		// alternative name
		$si->getIcon($url);
	}
	
	// retrieve icon from site; supply the bits as output
	$bits = $si->captureIcon($url);

	// convert icon bits to base64
	$b64dat = $si->asBase64($icon, $ext);
	
	// save icon bits locally
	$si->saveIcon($url, $ext, $icon_bits);
	
}

PageParser

Acquire useful information about a webpage or website, especially RSS feeds and meta content.

Usage

try {
	$pp = new PageParser();
	
	$pp->setUrl($url);
	$data = $pp->acquire($url = null);
	
	// does this site offer RSS/Atom feeds?  If so, iterate through them
	if ($pp->hasFeeds()) {
		$feed_url = $pp->getNextFeed();
	} 
	
	// site has meta keywords
	$pp->hasMetas();
	
	// all the information available for the site, as JSON 
	$json = $pp->asJson();
}

RateLimiter

The providers of many services ban consumers from executing too many requests in a given time period. This class "throttles" such API calls in a simple way.

NOTES: The first call to this object's pause() method is not penalized. The pause() algorithm generates a list of "spoilers" - where a large penalty is inserted (1-3 seconds) to further limit the access. These occur "rarely" relative to the MaxCalls rate, though.

Usage

try {
	$throttler = new RateLimiter(true);  // turn it ON
	$throttler->setMaxCallsPerSecond(5); // set rate limit

	$huge = big_long_list_of_requests(); //...your code
	foreach ($huge as $one) {
		$result = $this->my_retriever($one); // your action method
	}
}

private function my_retriever($data) {
	if ($this->throttler->isThrottling()) { $this->throttler->throttle(); }
		//...
		//... your code here
		//...
	}
}

Utilities

Small set of useful static methods.

Usage

// log to configured log file using PHP-defined error levels
Utilities::logger($str, $level = E_USER_NOTICE);

// so many hash algorithms to choose, odd assortment supported, figure it out
$bool = Utilities::hashAlgorithmExists($algo);

// reformat (many common) date strings into RFC822 format
$rfc822_str = Utilities::munge_date($str);

DataStore

A base (and abstract) class simplifying SQL databases and tables. The provided HttpResponse, MIMEType, CountryCode and CountryFlag classes are examples of usage.

Abstract Methods

abstract public function __construct($force = false);
abstract public function dump($to_file = null);
abstract protected function load_table();

try {

	$sql->setDatabase($db);
	$sql->setDatafile($file);
	$sql->setTable($table);
	$sql->setDbUser($u);
	$sql->setDbPassword($p);
	$sql->setDbHost($host)
	$sql->setDbPort($port);
	$sql->setDbOptions($options_array);

	// get data from specified column ($field), similar to $str_arg (using SQL LIKE)	
	$data = $sql->retrieveInfoLike($field, $str_arg);

	$sql->rebuild();
	$sql->reload();
	$column_names = $sql->columns();

	// output entire table, or selected columns, as JSON
 	$json = $sql->asJSON($columns = null) {
}

guardianproject / core_utilities

Maintainers

Details

README

Quick Reference

Installing the Utilities library

Configuring your installation

Initializing the Utilities Library

Testing Your Installation

Using this Library

API Description

CountryCode

CountryFlag

Cryptor

MIMEType

UUID

HttpResponse

URL

QRCode

SiteIcon

PageParser

RateLimiter

Utilities

DataStore

Abstract Methods