rattfieldnz/safe-urls

A laravel package to check URLs with Google's Safe Browsing API.

0.0.58 2019-08-14 09:38 UTC

Requires

• php: >=7.2
• ext-curl: *
• ext-json: *
• curl/curl: ^2.2
• illuminate/support: ~5

Requires (Dev)

• barryvdh/laravel-ide-helper: 2.6.*
• fzaninotto/faker: ~1.8
• mihaeu/test-generator: ^1.0
• mockery/mockery: ^1.2
• orchestra/testbench: ^3.8
• php-cs-fixer/phpunit-constraint-isidenticalstring: ^1.1
• phpspec/prophecy: ^1.8
• phpunit/php-code-coverage: ~6.1
• phpunit/phpunit: ~7.5
• sempro/phpunit-pretty-print: ^1.0
• squizlabs/php_codesniffer: ^3.4

MIT

• Robert Attfield

• google
• laravel
• safe-urls
• Safe Browsing API

README

A Laravel package to check URLs with Google's Safe Browsing API (look up).

Inspired by another similar package @ https://github.com/snipe/Safebrowsing.

Take a look at contributing.md to see a to do list.

• Installation
  • Update Config
  • Publish Config
  • Set Your Google Safebrowsing API Key
• Usage
  • Using Blade Syntax
  • Using Facades
• Example With Input and Output
• Test URLs
• Testing
• Contributing
• Security
• Credits
• License

Please Note:

This package requires that you have an active Google Safebrowsing API key. It absolutely will not work without one. It's free to create an API key .

Google also throttles API usage, so if you have a high-traffic site, you may want to build in a caching layer or something so you don't burn through your requests too quickly. You can keep an eye on your usage through the Google API console.

This project is not ready for use in production yet. When it is, there will be first major release (i.e. 1.0.0).

A Note About Google Safebrowsing API Results

During testing, there were a few times when the API showed some malware sites as 'safe', whereas in reality they weren't.

For example, running PHPUnit tests showed the sample sites below as being 'safe'; however, running the API in Postman produced expected results. I have not found a solution yet; however, any feedback / suggestions are welcome, as are pull requests etc. Below are links which I have read about this issue further:

• google/safebrowsing#30 (comment).
• https://stackoverflow.com/questions/41934692/google-url-safe-browsingv4-lookup-api-is-not-working.
• https://groups.google.com/forum/#!topic/google-safe-browsing-api/Z5FVGfBbl20
• https://stackoverflow.com/questions/54625443/google-safe-browsing-not-detecting-url-even-it-unsafe-url

Installation

Via Composer

$ composer require rattfieldnz/safe-urls

Update Your Config

For applications using Laravel =< 5.4

Open config/app.php and add:

RattfieldNz\SafeUrls\SafeUrlsServiceProvider::class,

to your providers array in config/app.php, and:

'SafeUrls' => RattfieldNz\SafeUrls\Facades\SafeUrlsFacade::class,

to your aliases array in config/app.php.

Publish the config

php artisan vendor:publish

This will add a safe-urls.php config file into your project's config folder.

Set Your Google Safebrowsing API Key

In your .env, add:

GOOGLE_API_KEY=YOUR-ACTUAL-API-KEY
GOOGLE_CLIENT_ID=your-client-id
GOOGLE_CLIENT_VERSION=1.0 (or your preferred number)
GOOGLE_CURL_TIMEOUT=10 (in seconds)

There are additional options in the config file that relate to what specific types of threats you want to check for, and what platforms you want to check on, but you only really need to worry about that if you want to check fewer things, as it's pretty inclusive already.

Usage

Using Blade Syntax

@if(isset(SafeUrls::check($urls, true)["response"]["matches"]))
    <p>There are {{ count(SafeUrls::check($urls, true)["response"]["matches"]) }} dangerous URLs.</p>
@else
    <p>No results were found</p>
@endif

Where $urls could be an array of URLs to check, perhaps passed through your Controller to a view.

Where true will return the results as an associative array.

false (or not having the second parameter) will return the results as a JSON-encoded string.

@if (SafeUrls::isDangerous('http://twitter.com/'))
    // do something if the url is flagged as suspicious
@else
    // hooray - it's not flagged!
@endif

Using Facades

SafeUrls::add(['http://ianfette.org']);
SafeUrls::add(['http://malware.testing.google.test/testing/malware/']);
SafeUrls::execute();
print('Status of the third URL is: '.SafeUrls::isDangerous('http://twitter.com/'));

Example with input and output

If the value of $urls was:

$urls = [
    'http://www.yahoo.com/'
    'http://www.google.com/'
    'http://malware.testing.google.test/testing/malware/'
    'http://twitter.com/'
    'http://ianfette.org'
    'https://github.com/'
    'https://testsafebrowsing.appspot.com/s/phishing.html'
    'https://testsafebrowsing.appspot.com/s/malware.html'
    'http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/'
    'http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/SOCIAL_ENGINEERING/URL/'
];   

SafeUrls::check($urls, true) Would return the following associative array:

[
    "status" => 200,
    "response" => 
        [
            "matches" => [
                [
                    "threatType" => "MALWARE",
                    "platformType" => "ANY_PLATFORM",
                    "threat" => [
                        "url" => "http://malware.testing.google.test/testing/malware/"
                    ],
                    "cacheDuration" => "300s",
                    "threatEntryType" => "URL"
                ],
                [
                    "threatType" => "SOCIAL_ENGINEERING",
                    "platformType" => "ANY_PLATFORM",
                    "threat" => [
                        "url" => "http://malware.testing.google.test/testing/malware/"
                    ],
                    "cacheDuration" => "300s",
                    "threatEntryType" => "URL"
                ],
                [
                    "threatType" => "SOCIAL_ENGINEERING",
                    "platformType" => "ANY_PLATFORM",
                    "threat" => [
                        "url" => "https://testsafebrowsing.appspot.com/s/phishing.html"
                    ],
                    "cacheDuration" => "300s",
                    "threatEntryType" => "URL"
                ],
                [
                    "threatType" => "MALWARE",
                    "platformType" => "ANY_PLATFORM",
                    "threat" => [
                        "url" => "https://testsafebrowsing.appspot.com/s/malware.html"
                    ],
                "cacheDuration" => "300s",
                "threatEntryType" => "URL"
            ],
            [
                "threatType" => "MALWARE",
                "platformType" => "ANY_PLATFORM",
                "threat" => [
                    "url" => "http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/"
                ],
                "cacheDuration" => "300s",
                "threatEntryType" => "URL"
            ],
            [
                "threatType" => "SOCIAL_ENGINEERING",
                "platformType" => "ANY_PLATFORM",
                "threat" => [
                    "url" => "http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/SOCIAL_ENGINEERING/URL/"
                ],
                "cacheDuration" => "300s",
                "threatEntryType" => "URL"
            ],
        ],
    ],
]

SafeUrls::check($urls) (or SafeUrls::check($urls, false)) Would return the following JSON-encoded string:

{  
   "status":200,
   "response":{  
      "matches":[  
         {  
            "threatType":"MALWARE",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"http:\/\/malware.testing.google.test\/testing\/malware\/"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         },
         {  
            "threatType":"SOCIAL_ENGINEERING",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"http:\/\/malware.testing.google.test\/testing\/malware\/"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         },
         {  
            "threatType":"SOCIAL_ENGINEERING",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"https:\/\/testsafebrowsing.appspot.com\/s\/phishing.html"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         },
         {  
            "threatType":"MALWARE",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"https:\/\/testsafebrowsing.appspot.com\/s\/malware.html"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         },
         {  
            "threatType":"MALWARE",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"http:\/\/testsafebrowsing.appspot.com\/apiv4\/ANY_PLATFORM\/MALWARE\/URL\/"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         },
         {  
            "threatType":"SOCIAL_ENGINEERING",
            "platformType":"ANY_PLATFORM",
            "threat":{  
               "url":"http:\/\/testsafebrowsing.appspot.com\/apiv4\/ANY_PLATFORM\/SOCIAL_ENGINEERING\/URL\/"
            },
            "cacheDuration":"300s",
            "threatEntryType":"URL"
         }
      ]
   }
}

Both outputs will depend on what options you have set in your config/safe-urls.php file.

Test URLs

Here are some handy test urls you can use while you're experimenting with this Laravel package.

• http://www.yahoo.com/ (OK)
• http://www.google.com/ (OK)
• http://malware.testing.google.test/testing/malware/ (Malware)
• http://twitter.com/ (OK)
• http://ianfette.org (Malware)
• https://github.com/ (OK)
• https://testsafebrowsing.appspot.com/s/phishing.html (Malware)
• https://testsafebrowsing.appspot.com/s/malware.html (Malware)
• http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/ (Malware)
• http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/SOCIAL_ENGINEERING/URL/ (Malware / Social Engineering)

Change log

Please see the changelog for more information on what has changed recently.

Testing

From inside the root folder of this package:

$ ./run_phpunit YOUR_GOOGLE_API_KEY

Replace YOUR_GOOGLE_API_KEY with your key. Get one by visiting https://developers.google.com/safe-browsing/v4/get-started.

Contributing

Please see contributing.md for details and a todolist.

Security

If you discover any security related issues, please email author email instead of using the issue tracker.

Credits

• Rob Attfield
• All Contributors

License

Please see the license file for more information.