Search by

toropyga / netcontent

Toropyga

Library for working with the network from Toropyga

Package info

github.com/Toropyga/NetContent

pkg:composer/toropyga/netcontent

Security

Advisories: 0

Aikido package health analysis

Transparency log

Statistics

Installs: 273

Dependents: 1

Suggesters: 0

Stars: 0

Open Issues: 0

v5.0.0 2026-09-06 12:07 UTC

Requires

Requires (Dev)

None

Suggests

None

Provides

None

Conflicts

None

Replaces

None

MIT 58911287bedee62fe91726683d8b72f6ce1d6ed2

Published on Packagist.org on 2026-09-06 12:08 UTC

This package is auto-updated.

Last update: 2026-09-06 12:08:46 UTC

README

Retrieving data from the Internet

License Version PHP

Contents

Overview

A library for retrieving content by URL (HTTP/HTTPS/FTP) with support for multiple transports, proxies, Basic authentication, SSRF protection, and HTML link rewriting for proxy-style page viewing.

The public API is concentrated in a single facade — Toropyga\NetContent — while all internal logic is split into independent modules under Toropyga\Net\*.

NetContent Class Features

This class can be used to retrieve HTML pages, images, and files from the Internet.

The class can connect to remote resources using the cURL library, direct socket communication, the standard file_get_contents() function, and fopen().

It supports authentication on proxy servers.

It allows arbitrary header parameters to be configured and sent.

It can log all operations.

Structure

src/
  NetContent.php                   Facade: the single entry point for consumers
  Net/Http/                        Request transports
    HttpClientInterface.php          Transport contract
    RequestOptions.php               Immutable request parameters
    ProxyConfig.php                  Immutable proxy parameters
    CurlHttpClient.php               cURL-based transport
    SocketHttpClient.php             Transport based on fsockopen/stream_socket_client
    FileGetContentsHttpClient.php    Transport based on file_get_contents()
    FopenHttpClient.php              Transport based on fopen()/fread()
    CurlOptionValidator.php          Type validation for setOPTcURL()
  Net/Security/                    SSRF protection
    SsrfGuard.php                    URL validation and resolution
    ResolvedTarget.php               Verified IP address for pinning
    SsrfViolationException.php       Exception raised when SSRF is detected
  Net/Html/HtmlRewriter.php        HTML link rewriting (mode 3)
  Net/Response/ResponseWriter.php  Output mode processing 1-4
  Net/Output/                      Abstraction over header()/echo
    OutputEmitterInterface.php
    PhpOutputEmitter.php             Actual output (default)
    BufferOutputEmitter.php          In-memory collection (tests, embedding)
  Net/Mime/MimeTypeResolver.php    File MIME type detection
  Net/Io/TempFileManager.php       Temporary file management
  Net/Logging/RequestLogger.php    Debug log + secret masking

Each module solves one specific task and does not know about the facade — NetContent only orchestrates their calls.

Installation

composer require toropyga/netcontent

Requires PHP 8.1+ and the curl extension (for the CURL transport, which is used by default).

Quick Start

use Toropyga\NetContent;

$nc = new NetContent();
$html = $nc->getContent('https://example.com/', 2); // 2 = return as a string

getContent() Function

The getContent() function is the main function of the class and accepts several parameters:

/*
 * @param string $url - URL of the requested resource
 * @param int $mode - processing mode for the received content
 * @param mixed $data - parameters passed in the request to the remote resource
 * @param string $savePath - path to the directory for saving the received file,
 *                           relative to the current directory or an absolute path
 *                           (if the directory does not exist, the class will try to create it)
 * @param string $saveName - name of the received file when saving
 * @return string|bool Response body (modes 2/5), true/false on success (modes 1/3/4),
 *                      false on error (SSRF block, connection failure, invalid URL)
 */
$nc->getContent(string $url, int $mode = 1, $data = '', string $savePath = '', string $saveName = '');

Example:

$nc->getContent('https://www.site.com', 4, 'files', 'index.html');

getContent() Modes

getContent(string $url, int $mode = 1, $data = '', string $savePath = '', string $saveName = '')

Mode ($mode) Behavior
1 Send Content-Type and the response body through header()/echo
2 Return the response body as a string
3 Same as mode 1, but with HTML links rewritten first (see below)
4 Save the response body to a file ($savePath/$saveName)
5 Return the response content "raw" as-is, without saving it to a temporary file

Returns false if the URL is blocked by SSRF protection, a connection error occurs, or the URL is invalid.

Configuration

Transports

$nc->setType('CURL');   // default
$nc->setType('SOCKET'); // manual HTTP construction over fsockopen/stream_socket_client
$nc->setType('FGC');    // file_get_contents() with a stream context
$nc->setType('FILE');   // fopen()/fread(), no proxy support

All four behave identically at the public API level: the same timeouts, TLS verification, SSRF pinning, and headers.

Proxy

$nc->setProxy('proxy.local', 3128, 'user', 'password'); // Configure proxy parameters (server, port, user, password)
$nc->setProxyUse(true); // Enable or disable use of the proxy server

Target Server Authentication

$nc->setUser('login', 'password');       // Basic authentication
$nc->setNCAuth('Bearer', $token, false); // or an arbitrary Authorization type on the remote server (auth type, auth key, add username and password to the URL)

SSRF Protection

Enabled by default. Blocks requests to private/reserved networks (RFC1918, loopback, link-local — including cloud metadata endpoints such as 169.254.169.254), CGNAT, forbidden schemes, and forbidden ports. The verified IP is pinned for CURL and SOCKET transports via CURLOPT_RESOLVE/direct connection by IP, and for FGC/FOPEN by replacing the host with the IP in the URL. This prevents DNS rebinding between validation and the actual connection.

$nc->setSsrfProtection(false);            // fully disable (trusted scenarios only)
$nc->setAllowPrivateNetworks(true);       // explicitly allow private networks
$nc->setSsrfAllowedSchemes(['https']);
$nc->setSsrfDeniedPorts([22, 3306, 6379]);

Limitation: when using a proxy (setProxyUse(true)), pinning does not apply — the proxy performs DNS resolution, not this library. URL validation is still performed (defense in depth), but full protection against rebinding in this case depends on the proxy's own policy.

Additional Settings

$nc->setNCTimeOut($time_in_seconds) // Set the server response timeout
$nc->setMethod('GET|POST')          // Set the data transfer method when connecting to the requested URL (GET or POST)
$nc->setHeaderCURL(true|false)      // Set whether response headers should be included when using CURL
$nc->setHeaders($header, $value)    // Set additional headers used when connecting
$nc->setOPTcURL($option, $value)    // Set configuration parameters for the cURL library
$nc->setSecure(true|false);         // true = disable TLS certificate verification

Constants

The class supports configuration through optional predefined constants:

NET_DEBUG               - enable/disable debugging
NET_TYPE                - type of connection used
NET_USE_PROXY           - whether to use a Proxy server
NET_PROXY_ADDRESS       - Proxy server address
NET_PROXY_PORT          - Proxy server port
NET_PROXY_USER          - Proxy server username
NET_PROXY_PASSWD        - Proxy server user password
NET_TIMEOUT             - server response timeout
NET_METHOD              - data transfer method when connecting to the requested URL (GET or POST)
NET_PROTOCOL            - default interaction protocol (http, https, etc.)
NET_NOT_SECURITY        - use an insecure connection in the cURL module
NET_LOG_NAME            - log file name

HTML Rewriting (Mode 3)

In mode 3, the HTML response is processed by HtmlRewriter: relative and protocol-relative links in <a>, <img>, <script>, <link>, background, and similar elements are converted to absolute URLs. <a>/<img> links also receive a local handler prefix, making it convenient to build a simple proxy-style page viewer through your own server.

Capturing Output Instead of header()/echo

Modes 1 and 3 normally print the response directly. If NetContent is used as a library inside an application that needs to build the HTTP response itself (or in tests where header() would cause headers already sent), use BufferOutputEmitter:

use Toropyga\Net\Output\BufferOutputEmitter;

$emitter = new BufferOutputEmitter();
$nc->setOutputEmitter($emitter);
$nc->getContent($url, 1);

$body = $emitter->body();
$headers = $emitter->headers();

Logging

$nc->setDebug(true);
$nc->setLogger($psr3Logger); // any Psr\Log\LoggerInterface

// or retrieve the log manually:
// true - entire log, false - last entry
$logs = $nc->getLogs(true); // ['log' => [...], 'file' => 'net.log']

Passwords, tokens, and other secrets are masked (***) before being written to the log.

Working with "Raw" cURL Options

$nc->setOPTcURL(CURLOPT_FOLLOWLOCATION, true);

The value is checked against the expected option type; if it does not match or the option is unknown, the call returns false and logs the reason (when setDebug(true) is enabled).

Warning: enabling CURLOPT_FOLLOWLOCATION makes cURL follow redirects automatically, including redirects to other hosts — SsrfGuard checks only the original URL, so a redirect to an internal address would bypass the protection. Keep redirects disabled (the default) unless they are absolutely necessary for the task.

Additional Functions

Returns the path to the last saved file:

$path = $nc->getLastSavedPath();

Determines the file MIME TYPE:

// $filename - path to the file
$mime = $nc->get_mime_content_type($filename);

Testing

Files are installed through Composer (composer install). For a quick syntax check without installing dependencies:

find src -name "*.php" -exec php -l {} \;