README

based on kassner/log-parser primarily designed to parse web access logs, Parselog extends to other logs like web error, syslog, fail2ban, ... (in progress)

• Features
• Requirments
• Api Documentation
• License

Features

• Generic customizable log parser
• Predefined software log parsers: ✓ Apache Access, ✓ Apache Error, ✓ Fail2ban, ✓ Syslog
• IPv4 & IPv6 recognition patterns

Requirments

• PHP >= 7.1
• Composer (for install)

Api Documentation

draft...

• LogParser overview
  • LogParser class
  • Basic usage
  • Use custom LogEntry
• Software parsers
  • ApacheAccessLogParser
  • ApacheErrorLogParser
  • SyslogParser
  • Fail2BanLogParser

LogParser overview

The library comes with a generic LogParser class you can configure from scratch to parse something, and predefined software parsers class you can use with no or less configuration.

LogParser class

Basic usage

To use the \Kristuff\Parselog\LogParser class, you need to define the log format, then parse data using LogParser::parse() method.

$data = 'fooValue2 123456'; // the data to parse
$parser = new \Kristuff\Parselog\LogParser(); // create a LogParser instance
$parser->setFormat('(?P<foo>(fooValue1|fooValue2)) (?P<bar>[0-9]+)'); // set format including pattern
$entry = $parser->parse($data);// parse data

The LogParser::parse() method returns a \Kristuff\Parselog\Core\LogEntry object. That $entry object will hold all data parsed.

echo $entry->foo; // fooValue1
echo $entry->bar; // 123456

If the line does not match the defined format, a \Kristuff\Parselog\FormatException will be thrown.

Define log format

Defining the log format can be done in different ways. In section above, we defined the full pattern using the LogParser::setFormat() method. To define the log format more easily, you can use the LogParser::addPattern() or LogParser::addNamedpattern() methods.

This 3 blocks of code produce the same result:

// set format including pattern
$parser->setFormat('(?P<foo>(fooValue1|fooValue2)) (?P<bar>[0-9]+)');

// OR add patterns, then define format
$parser->addPattern('%1', '(?P<foo>(fooValue1|fooValue2))'); 
$parser->addPattern('%2', '(?P<bar>[0-9]+)'); 
$parser->setFormat('%1 %2');

// OR add named patterns, then define format
$parser->addNamedPattern('%1', 'foo', '(fooValue1|fooValue2)'); 
$parser->addNamedPattern('%2', 'bar', '[0-9]+'); 
$parser->setFormat('%1 %2');

TODO $parser->addNamedPattern with optional column

Use custom LogEntry

By default, the LogParser::parse() method returns a \Kristuff\Parselog\Core\LogEntry object. To use your own entry class, you will have to:

• create two new classes, your entry object that implements \Kristuff\Parselog\Core\LogEntryInterface interface and a factory, that implements \Kristuff\Parselog\Core\LogEntryInterface interface and that is responsible of creating it:

class MyEntry implements \Kristuff\Parselog\Core\LogEntryInterface
{
}

class MyEntryFactory implements \Kristuff\Parselog\Core\LogEntryFactoryInterface
{
    public function create(array $data): \Kristuff\Parselog\Core\LogEntryInterface
    {
        // @TODO implement your code here to return a instance of MyEntry
    }
}

• and then provide the factory as the second argument to the LogParser or SoftwareLogParser constructor:

$factory = new MyEntryFactory();
$parser = new \Kristuff\Parselog\Sofware\ApacheAccessLogParser(null, $factory);
$entry = $parser->parse('193.191.216.76 - www-data [27/Jan/2014:04:51:16 +0100] "GET /wp-content/uploads/2013/11/whatever.jpg HTTP/1.1" 200 58678');

`$entry` will be an instance of `MyEntry`.

Software parsers

All software parsers extand the \Kristuff\Parselog\Software\SoftwareLogParser class, contain software configuration and provide helper functions to get the default log files, to get the defaults formats, ... You can create sotfware parser in two ways:

• Use the dedicated existing class in \Kristuff\Parselog\Software like ApacheAccessLogParser. Current implementation are:

  • ApacheAccessLogParser
  • ApacheErrorLogParser
  • Fail2BanLogParser
  • SyslogParser

• Or create a SoftwareLogParser instance from the \ Kristuff\Parslog\LogParserFactory::getParser() method:

  use  Kristuff\Parslog\LogParserFactory;
  
  $parser = LogParserFactory::getParser(LogParserFactory::TYPE_APACHE_ACCESS);

  The \ Kristuff\Parslog\LogParserFactory::getParser() method takes the logtype as argument. Valide values (string) are:

  • LogParserFactory::TYPE_APACHE_ACCESS // "apache_access"
  • LogParserFactory::TYPE_APACHE_ERROR // "apache_error"
  • LogParserFactory::TYPE_APACHE_FAIL2BAN // "fail2ban"
  • LogParserFactory::TYPE_APACHE_SYSLOG // "syslog"

The \Kristuff\Parselog\Software\SoftwareLogParser class extends the \Kristuff\Parselog\LogParser and come with aditionally methods:

ApacheAccessLogParser

Create an ApacheAccessLogParser instance:

// use default format and entry factory
$parser = new \ Kristuff\Parslog\Software\ApacheAccessLogParser(); 

// use explicit format and default entry factory
$parser = new \ Kristuff\Parslog\Software\ApacheAccessLogParser('%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"'); 

Create an SoftwareLogParser instance from factory:

use  Kristuff\Parslog\LogParserFactory;

$parser = LogParserFactory::getParser(LogParserFactory::TYPE_APACHE_ACCESS);

Apache access log format

The default ApacheAccessLogParser format is %h %l %u %t "%r" %>s %O. You can retreive this format using the constant ApacheAccessLogParser::FORMAT_COMMON. Other registered formats are following:

ApacheAccessLogParser::FORMAT_COMBINED       // %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"
ApacheAccessLogParser::FORMAT_COMBINED_VHOST // %v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"
ApacheAccessLogParser::FORMAT_COMMON_VHOST   // %v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"
ApacheAccessLogParser::FORMAT_REFERER        // %{Referer}i 
ApacheAccessLogParser::FORMAT_AGENT          // %{User-Agent}i 

The library works with Apache access log format since version 2.2. Here is the full list of log format strings supported by Apache 2.4, and whether they are supported by the library:

ApacheErrorLogParser

Basic usage

Create an ApacheErrorLogParser instance:

// use default format and entry factory
$parser = new \ Kristuff\Parslog\Software\ApacheAccessLogParser(); 

// use explicit format and default entry factory
$parser = new \ Kristuff\Parslog\Software\ApacheAccessLogeError('YOUR FORMAT'); 

Create an SoftwareLogParser instance from factory:

use  Kristuff\Parslog\LogParserFactory;
$parser = LogParserFactory::getParser(LogParserFactory::TYPE_APACHE_ERROR);

Apache error log format

The default ApacheErrorLogParser format is [%{u}t] [%l] [pid %P] [client %a] %F: %E: %M (format can include brackets). You can retreive this format using the constant ApacheErrorLogParser::FORMAT_DEFAULT_APACHE_2_4. Other registered formats are following:

ApacheErrorLogParser::FORMAT_DEFAULT_APACHE_2_2   //[%t] [%l] [client %a] %F: %E: %M    
ApacheErrorLogParser::FORMAT_DEFAULT_APACHE_2_4   //[%{u}t] [%l] [pid %P] [client %a] %F: %E: %M    
ApacheErrorLogParser::FORMAT_MPM_APACHE_2_4       //[%{u}t] [%-m:%l] [pid %P] [client %a] %F: %E: %M    
ApacheErrorLogParser::FORMAT_MPM_TID_APACHE_2_4   //[%{u}t] [%-m:%l] [pid %P:tid %T] [client %a] %F: %E: %M    

The library supports Apache error log format version 2.2 and 2.4 with same parser. Here is a partial list of log format strings supported by Apache 2.4, and whether they are supported by the library:

SyslogParser

Basic usage

Create an SyslogParser instance:

// use default format and entry factory
$parser = new \ Kristuff\Parslog\Software\SyslogParser(); 

Create an SoftwareLogParser instance from factory:

use Kristuff\Parslog\LogParserFactory;
$parser = LogParserFactory::getParser(LogParserFactory::TYPE_SYSLOG);

Syslog format

The default SyslogParser format is %t %h %s%p: %m. The fields are detailed below:

Fail2BanLogParser

Basic usage

Create an Fail2BanLogParser instance:

// use default format and entry factory
$parser = new \ Kristuff\Parslog\Software\Fail2banLogParser(); 

Create an SoftwareLogParser instance from factory:

use  Kristuff\Parslog\LogParserFactory;
$parser = LogParserFactory::getParser(LogParserFactory::TYPE_FAIL2BAN);

Fail2Ban log format

Here are some log line examples from Fail2ban (version 0.10.2):

2020-08-15 10:11:15,839 fail2ban.actions        [6924]: NOTICE  [_apache_hack] Ban 1.2.3.4
2020-08-14 10:44:57,101 fail2ban.utils          [536]: Level 39 7f4d265d09f0 -- returned 1

The default Fail2BanLogParser format is %t %s %p %l %j %m. The fields are detailed below:

Limitations / Known issues

• Field time does not register milliseconds

License

The MIT License (MIT)

Copyright (c) 2017-2020 Kristuff

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.