simbiat/lodestone-parser

FFXIV Lodestone parser in PHP

1.1.7+20221010 2022-10-10 09:48 UTC

README

This project is PHP library for parsing data directly from the FFXIV Lodestone website initially based on one developed by @viion, but now completely rewritten. Previously was its branch.

The goal is to provide an extremely fast and lightweight library, it is built with the purpose of parsing as many characters as possible, key being: low memory, and micro-timed parsing methods.

Notes

  • This library parses the live Lodestone website. This website is based in Tokyo.
  • This library is built in PHP 8 minimum, please use the latest as this can increase

What's different?

This is what's different from original library from @viion:

  • It has different code structure, that aims at reduction of rarely used or unnecessary functions and some standardization.
  • Using regex instead of full HTML parsing for extra speed (and arrays instead of objects as result). It does not mean, that this will always be faster than using Symphony-based functions but will be true on average.
  • More filters for your search queries.
  • Return more potentially useful information where possible.
  • Attempt at multilingual support. Some filters even support actual "names" used on Lodestone (instead of just IDs).
  • Ability to "link" different types of entities, requesting several pages in one object. For example, you can get both Free Company and its members' details in same object.

Settings

It's possible to set your own UserAgent used by CURL: simply use ->setUseragent('useragent')

It's also possible to change LodeStone language by ->setLanguage('na'). Accepted language values are na, eu, jp, fr, de

It's possible to utilize Benchmarking to get parsing times for each iteration by ->setBenchmark(true)

Error handling

In the new concept fatal errors generally can happen only during HTTP requests. In order not to break "linking" function, they are handled softly in the code itself and are reported to ->errors and ->lasterror arrays. In essence, when an error occurs you will simply get an empty result for specific entity and it will not be added to output.

To get last error you can use ->getLastError($close). For list of all errors - -getErrors($close). $close is an expected boolean, that, if set to true, will close the cURL handle. It is set to false by default, based on assumption, that you are using these only for some kind of validation.

All pages

All parsers accepting page number support value of 0, which will return all pages (that is run the respective parser recursively). Default value for page is set to 1 to limit resources used.

Test script

There is a \Simbiat\LodestoneTest class to test run all the available functions in some general scenarios. Run it to get samples of output formatting and timings for each type of test in a table format. Note, that the last 2 tests are 'error tests', so their results are purposefully reversed for the report's consistency. Additionally, achievements' test is purposefully ran with details set to true and Free Company members in All pages mode, because of this their benchmark results will be presented as list of timings.

Getting results

To get results of parsers (listed below) you need to run ->getResult($close), which will return the array with the results. $close is an expected boolean, that, if set to true, will close the cURL handle. It is set to true by default, based on assumption, that getting results will be your last action.

NOTICE: if you are not calling ->getResult($close) at all or are calling it only with false, it's recommended to explicitly unset the object you are using, unless you are using to free up resources.

Parsers

Function Parameters (in required order) Return key Description
Characters
getCharacter $id - id of character. characters[$character], where $character is id of character returned with respective details as an array. Returns character details.
getCharacterJobs $id - id of character. characters[$id]['jobs'], where $id is id of character. Returns character jobs' details.
getCharacterFriends
  • $id - id of character.
  • int $page = 1 - characters' page. Defaults to 1.
characters[$id]['friends'][$character], where $id is id of character and $character is id of friends returned with respective details as an array. Returns character's friends.
getCharacterFollowing
  • $id - id of character.
  • int $page = 1 - characters' page. Defaults to 1.
characters[$id]['followed'][$character], where $id is id of character and $character is id of followed characters returned with respective details as an array. Returns characters followed by selected one.
getCharacterAchievements
  • $id - id of character.
  • $achievementId = false - id of achievement. Required if you want to search for specific achievement.
  • $kind = 1 - category of achievement. Acts as subcategory if $category is true. Multilingual. If `0` is sent all categories will be retrieved.
  • bool $category = false - switch to turn $kind into subcategory.
  • bool $details = false - switch to grab details for all achievements in category. Be careful, since this will increase runtimes proportionally to amount of achievements.
  • bool $only_owned = false - flag to return only owned achievements. Returns everything by default.
characters[$character]['achievements'][$achievement], where $character is id of character and $achievement is id of achievement returned with respective details as an array. Returns character's achievements, if the are public.
Groups
getFreeCompany $id - id of Free Company. freecompanies[$freecompany], where $freecompany is id of Free Company returned with respective details as an array. Returns information about Free Company without members.
getFreeCompanyMembers
  • $id - id of Free Company.
  • int $page = 1 - members' page. Defaults to 1
freecompanies[$freecompany][$character], where $freecompany is id of Free Company and $character is id of each member returned with respective details as an array. Returns requested members' page of the Free Company.
getLinkshell
  • $id - id of Linkshell (numeric) or Crossworld Linkshell (alfa-numeric).
  • int $page = 1 - members' page. Defaults to 1
linkshells[$linkshell], where $linkshell is id of linkshell returned with respective details as an array. Returns requested member's page of the Linkshell and general information.
getPvPTeam $id - id of PvP Team. pvpteams[$pvpteam], where $pvpteam is id of PvP Team returned with respective details as an array. Returns general information and members of PvP Team.
Ranking
getFeast
  • int $season = 1 - number of season to get results for. Defaults to 1.
  • string $dcGroup = '' - server name to filter. Defaults to empty string, meaning no filtering.
  • string $rank_type = 'all' - type of rank to filter. Defaults to all, meaning no filtering. Multilingual.
feast[$season][$character], where $season is the value passed at call and $character is id of each character returned with respective details as an array. Returns The Feasts rankings for requested season, server and/or rank.
getDeepDungeon
  • int $id = 1 - id of Deep Dungeon as per Lodestone. 1 stands for 'Palace of the Dead', 2 stands for 'Heaven-on-High'. Defaults to 1.
  • string $dcGroup = '' - server name to filter. Defaults to empty string, meaning no filtering.
  • string $solo_party = 'party' - 'party' or 'solo' rankings to get. Defaults to party, same as Lodestone.
  • string $subtype = 'PLD' - job to filter. Used only if $solo_party is set to solo. Expects common 3-letter abbreviations and defaults to PLD, same as Lodestone.
deepdungeon[$id]['party'][$character] or deepdungeon[$id]['solo'][$subtype][$character], where $id is id of the dungeon, $subtype is common 3-letter abbreviation of the respective job and $character is id of each character returned with respective details as an array. Returns ranking of respective Deep Dungeon.
getFrontline
  • string $week_month = 'weekly' - type of ranking. Defaults to 'weekly'.
  • int $week = 0 - number of week (YYYYNN format) or month (YYYYMM format). Defaults to 0, that is current week or month.
  • string $dcGroup = '' - data center name to filter. Defaults to empty string, meaning no filtering.
  • string $worldname = '' - server name to filter. Defaults to empty string, meaning no filtering.
  • int $pvp_rank = 0 - minimum PvP rank to filter. Defaults to 0, meaning no filtering.
  • int $match = 0 - minimum number of matches to filter. Defaults to 0, meaning no filtering.
  • string $gcId = '' - Grand Company to filter. Defaults to empty string, meaning no filtering. Multilingual
  • string $sort = 'win' - sorting order. Accepts 'win' (sort by number of won matches), 'match' (sort by total number of matches) and 'rate' (sort by winning rate). Defaults to 'win'.
frontline['weekly'][$week][$character] or frontline['monthly'][$month][$character], where $week and $month is identification of request week or month and $character is id of each character returned with respective details as an array. Returns Frontline rankings for selected period.
getGrandCompanyRanking
  • string $week_month = 'weekly' - type of ranking. Defaults to 'weekly'.
  • int $week = 0 - number of week (YYYYNN format) or month (YYYYMM format). Defaults to 0, that is current week or month.
  • string $worldname = '' - server name to filter. Defaults to empty string, meaning no filtering.
  • string $gcId = '' - Grand Company to filter. Defaults to empty string, meaning no filtering. Multilingual
  • int $page = 1 - number of the page to parse. Defaults to 1.
GrandCompanyRanking['weekly'][$week][$character] or GrandCompanyRanking['monthly'][$month][$character], where $week and $month is identification of request week or month and $character is id of each character returned with respective details as an array. Returns Grand Company rankings for selected period.
getFreeCompanyRanking
  • string $week_month = 'weekly' - type of ranking. Defaults to 'weekly'.
  • int $week = 0 - number of week (YYYYNN format) or month (YYYYMM format). Defaults to 0, that is current week or month.
  • string $worldname = '' - server name to filter. Defaults to empty string, meaning no filtering.
  • string $gcId = '' - Free Company to filter. Defaults to empty string, meaning no filtering. Multilingual
  • int $page = 1 - number of the page to parse. Defaults to 1.
FreeCompanyRanking['weekly'][$week][$character] or FreeCompanyRanking['monthly'][$month][$character], where $week and $month is identification of request week or month and $character is id of each character returned with respective details as an array. Returns Free Company rankings for selected period.
Search
searchDatabase
  • string $type - mandatory type of entity to search for on Lodestone taken from https://eu.finalfantasyxiv.com/lodestone/playguide/db/$type.
  • int $category = 0 - optional ID of category taken from https://eu.finalfantasyxiv.com/lodestone/playguide/db/achievement/?category2=$category.
  • int $subcategory = 0 - optional ID of subcategory taken from https://eu.finalfantasyxiv.com/lodestone/playguide/db/achievement/?category2=2&category3=$subcategory. Will be ignored if category is not set.
  • string $search = '' - optional string to search for.
  • int $page = 1 - number of the page to parse. Defaults to 1.
database[$type][$entityid], where $type is type of entities searched, and $entityid is ID of the entity in the database returned with respective details as an array. Returns array fo entities from respective search function with array keys being respective entity's id on Lodestone.
searchCharacter
  • string $name = '' - optional name to search.
  • string $server = '' - optional server name to filter.
  • string $classJob = '' - optional filter by class/job. Supports types of jobs and common 3-letter abbreviations.
  • string $race_tribe = '' - optional filter by tribe/clan. Multilingual.
  • $gcId = '' - optional filter by Grand Company affiliation. Accepts singular string or an array of such. Multilingual.
  • $blog_lang = '' - optional filter by character language. Accepts same variables as for language setting. Accepts singular string or an array of such.
  • string $order = '' - optional sorting order. Refer to Converters.php for possible values.
  • int $page = 1 - number of the page to parse. Defaults to 1.
characters[$character], where $character is id of each character returned with respective details as an array.
searchFreeCompany
  • string $name = '' - optional name to search.
  • string $server = '' - optional server name to filter.
  • int $character_count = 0 - filter by Free Company size. Supports same counts as Lodestone: 1-10, 11-30, 31-50, 51-. Anything else will result in no filtering.
  • $activities = '' - optional filter by Company activities. Accepts singular string or an array of such. Multilingual.
  • $roles = '' - optional filter by seeking roles. Accepts singular string or an array of such. Multilingual.
  • string $activeTime = '' - optional filter by active time. Multilingual.
  • string $join = '' - optional filter by recruitment status. Multilingual.
  • string $house = '' - optional filter by estate availability. Multilingual.
  • $gcId = '' - optional filter by Grand Company affiliation. Accepts singular string or an array of such. Multilingual.
  • string $order = '' - optional sorting order. Refer to Converters.php for possible values.
  • int $page = 1 - number of the page to parse. Defaults to 1.
freecompanies[$freecompany], where $freecompany is id of each Free Company returned with respective details as an array.
searchLinkshell
  • string $name = '' - optional name to search.
  • string $server = '' - optional server name to filter.
  • int $character_count = 0 - filter by Linkshell size. Supports same counts as Lodestone: 1-10, 11-30, 31-50, 51-. Anything else will result in no filtering.
  • string $order = '' - optional sorting order. Refer to Converters.php for possible values.
  • int $page = 1 - number of the page to parse. Defaults to 1.
  • bool $crossworld = false - whether we are searching for regular or crossworld Linkshell. Searching for regular ones by default.
linkshells[$linkshell], where $linkshell is id of each linkshell returned with respective details as an array.
searchPvPTeam
  • string $name = '' - optional name to search.
  • string $server = '' - optional server name to filter.
  • string $order = '' - optional sorting order. Refer to Converters.php for possible values.
  • int $page = 1 - number of the page to parse. Defaults to 1.
pvpteams[$pvpteam], where $pvpteam is id of each PvP Team returned with respective details as an array.
News
getLodestoneNews news Returns news as seen on main page of Lodestone.
getLodestoneTopics int $page=1 - number of the page to parse. Defaults to 1. topics Return respective news subcategories.
getLodestoneNotices notices
getLodestoneMaintenance maintenance
getLodestoneUpdates updates
getLodestoneStatus status
Special
getLodestoneBanners banners Returns banners from Lodestone.
getWorldStatus bool $worldDetails=false - whether to show detailed status of worlds or not. Defaults to false. worlds Returns alphabet sorted array with worlds (servers) names as array keys and status (online/offline) as values. In detailed mode shows online status, maintenance status, whether world is preferred or congested and whether world can have new characters as boolean values.