README

Introduction

This plugin permits to generate REST modules bound to Doctrine models. It allows to easily create REST webservices, and provides an extensible framework for data exchange. Here are some key features :

REST module generation "à la admin-generator"
easy-to-customize generator.yml configuration file
validation of the parameters passed to the service using symfony validators
serialization as JSON
possibility to embed related models, in retrieval as well as in creation requests
possibility to embed extra fields
ability to limit the number of results, with or without pagination
support for constraints unions (ie., http://api.example.org/city?city_id=12,13,14)
hookable through events and filters
abstract and replaceable objects serialization
full HTTP support (GET, POST, PUT, DELETE)
functional test for each module generated

How to install

go to your project's root

Install the plugin:

 ./symfony plugin:install --stability=beta sfDoctrineRestGeneratorPlugin

clear the cache:
```
 ./symfony cc
```
alternatively, you might prefer to install this plugin as a Subversion dependancy. In this case, here is the repository: http://svn.symfony-project.com/plugins/sfDoctrineRestGeneratorPlugin. There is also a Git mirror on GitHub : https://github.com/xavierlacot/sfDoctrineRestGeneratorPlugin

Usage

REST module generation

Generating a REST module is pretty straightforward:

   ./symfony doctrine:generate-rest-module  APPLICATION MODULE MODEL

This will create a module named "MODULE" in the application "APPLICATION", and this module will be configured to expose the "MODEL" model through a REST-style service.

What is generated

Let suppose we have the following model :

    Post:
      actAs:                      [ Timestampable ]
      columns:
        post_category_id:         integer(4)
        created_by:               integer
        title:                    {type: string(128), notnull: true}
        summary:                  {type: string(255), notnull: true}
        body:                     clob
      relations:
        CreatedBy:                { class: sfGuardUser, onDelete: SET NULL, local: created_by, foreign: id, foreignAlias: CreatedPost }
        PostCategory:             { class: PostCategory, onDelete: SET NULL, local: post_category_id, foreign: id }

    PostCategory:
      columns:
        id:                        { type: integer(4), primary: true, autoincrement: true }
        name:                      { type: string, size: 100, notnull: true, unique: true }
        description:               clob
        is_enabled:                { type: boolean, default: true }

If we want to expose the model "Post" through a REST API, we will simply type the command:

    ./symfony doctrine:generate-rest-module  api post Post

This will generate:

a new route in the routing.yml file of the "api" application :

  post:
    class:   sfObjectRouteCollection
    options:
      model:   Post
      actions: [ create, delete, list, show, update ]
      module:  post
      column:  id
      format:  xml

a "post" RESTgen module, in apps/api/modules/post, with four sub-directories:
- actions: contains a "postActions" class, which extends a on-the-fly generated "autopostActions" class,
- config: contains the files generator.yml and view.yml (see the chapter "Service configuration" for explanations on how to configure the generated module),
- lib: contains an empty "postGeneratorConfiguration" class, which extends a on-the-fly generated "BasePostGeneratorConfiguration" class,
- templates
after a first request has been made to the REST module, the cache directory will contain the code of the generated module, and particularly the code of the "autopostActions" class, which you should check in order to understand the way the plugin works.

You should be able to see your posts as a JSON feed at http://api.example.com/post.json

If this is not the case, please check in the routing.yml file that the route name comes with a minus "p". If you want to change the uri at which the API is exposed, simply change the route name, and clear the cache of your project. The following configuration will expose the API at http://api.example.com/myPostAPI.json:

     myPostAPI:
       class:   sfObjectRouteCollection
       options:
         model:   Post
         actions: [ create, delete, list, update ]
         module:  post
         column:  id
         format:  xml

Main configuration

Before configuring the content of the response of the webservice, the first most important configuration steps must be undergone. They address some general concerns which impact security, stability and define what the webservice is supposed to do:

which types of operations does the webservice support? Only fetching objects, or also creating/updating/deleting it?
how does one access to the webservice? Which are the main security guidelines?

Allowed operations

The REST web service supports five different operations:

getting a list of items ("list"): http://api.example.com/post.json (GET method),
getting a single item ("show"): http://api.example.com/post/1.json (GET method)
creating a new item ("create"): http://api.example.com/post.json (POST method),
updating an existing item ("update"): http://api.example.com/post/123.json (PUT method, the value "123" must be replaced with the primary key of the object to update),
deleting an existing item ("delete"): http://api.example.com/post/123.json (DELETE method, the value "123" must be replaced with the primary key of the object to delete),

Each of these operations can be allowed or forbidden in the routing file, with the actions key: only enable the ones that you want to use.

Security guidelines

You should never forget that exposing a webservice with write access may harm your data. Because of a miss of attention in the way your webservice is secured, you could lose some important data, or have it be altered while it shouldn't.

First, be sure to only allow the strictly required operations. If the client do not have to delete items with the webservice, then disable this action in the routing.yml file.

Second, consider a way to make your webservice more secure:

use SSL whenever possible, so that the posted data do not get intercepted and altered by a third party (man in the middle),
use HTTP authentication,
use a stronger / more extensible authentication system (OAuth for example),
deliver unique API keys to your clients, and check the usage that they do of the API.

Detailed service configuration

As for symfony's admin-generator, the REST generator generates code on-the-fly, depending on the configuration done in the generator.yml file.

Here is the default content of the generator.yml file:

    generator:
      class: sfDoctrineRestGenerator
      param:
        model_class:   Post

        config:
          default:
    #        fields:                                # list here the fields.
    #          created_at:                  { date_format: 'Y-m-d\TH:i:s', tag_name: 'created' }      # for instance
    #        separator:                     ','     # separator used for multiple filters
          get:
    #        additional_params:             []      # list here additional params names, which are not object properties
    #        display:                       []      # list here the fields to render in the response
    #        embed_relations:               []      # list here relations to embed in the response
    #        embedded_relations_hide:
    #          Category:                    [id]    # you can hide fields inside a certain embedded relation
    #        global_additional_fields:      []      # list here additionnal calculated global fields
    #        hide:                          [id]    # list here the fields you don't want to expose
    #        max_items:                     0       # uncomment to fix an absolute limit to the number of items in the response
    #        object_additional_fields:      []      # list here additional calculated fields
    #        pagination_enabled:            false   # set to true to activate the pagination
    #        pagination_custom_page_size:   false   # set to true to allow the client to pass a page_size parameter
    #        pagination_page_size:          100     # the default number of items in a page
    #        sort_custom:                   false   # set to true to allow the client to pass a sort_by and a sort_order parameter
    #        sort_default:                  []      # set to [column, asc|desc] in order to sort on a column
    #        filters:                               # list here the filters
    #          id:
    #            compare:                   { less: , greaterEqual: moreThan } # generate filters idLess and idMoreThan
    #          created_at:                  { date_format: 'd-m-Y', multiple: true }  # for instance
          create:
    #        disable_validators:            [ created_at ] # list here validators that should not be generated

The different possible parameters, commented in the previous sample, are detailed in the following chapters.

model_class

The model_class parameters defines the name of the Doctrine model the REST module is bound to.

default

The default option contains several general configuration directives:

fields

The fields option contains, for each of the fields of the model, an array of decoration options that are used during the (de-)serialization. It might be:

date_format: the date format to use when formatting the field. This must be a format acceptable for the date() function,
tag_name: the tag name to use for displaying this field. For instance, you might want to associate the title of the post to the key "post_title", and not "title".
type: the type to use for displaying this field. All php cast keywords are supported, null values are retained.

separator

The separator to use in url when passing objects primary keys. The generated module allows to require several resources identified by their ids: http://api.example.com/post/?id=12,17,19

get

The get option lists several options specific to the "get" operation:

additional_params

The additional_params option allows to define an array of parameter names, which the webservice will accept.

The validation of the parameters in the generator is rather strict, and for every unrecognised parameter passed to the service, the generator will launch an exception. The option allows not to launch this exception for certain parameter types, even if these parameters do not actually get used by the generator.

The purpose of this parameter is to allow third-party params to be passed to the service. For instance, you might want to pass a "token" or "api_key" parameter, which could then be used to check if the client is allowed to use the service.

display

The display option contains the list of the fields to output in the XML or JSON feed. For example with the previously defined "Post" model, you can choose to only display the title and the author's id by changing this parameter:

        config:
          get:
            display:                       [ title, author_id ]

If this option is left empty, all the fields of the model will be rendered.

You might also be interested in the hide option, which allows to hide some fields of the model.

embed_relations

The embed_relations options contains the list of the Doctrine relations to be embedded. It might be 1-n or n-n relations, which content will be embedded in each object. Here is a valid configuration for our "Post" model:

        config:
          get:
            embed_relations:                       [ PostCategory ]

This configuration will produce a feed like:

...
<Post>
  <Id>1</Id>
  <PostCategoryId>2</PostCategoryId>
  <CreatedBy>26</CreatedBy>
  <Title>Here the title of my post</Title>
  <Summary>Here the summary of my post</Summary>
  <Body>Here the body of my post</Body>
  <PostCategory>
    <Id>2</Id>
    <Name>Name of the category</Name>
    <Description>Description of the category</Description>
    <IsEnabled>1</IsEnabled>
  </PostCategory>
</Post>
...

Several things to consider:

You cannot define the fields to render in the related objects, but you can hide certain with embedded_relations_hide option.
The response contains both the PostCategoryId field and the PostCategory.Id fields. You can save some bytes by using the display or the hide options, in order to remove the PostCategoryId field.

embed_relations_custom

The embed_relations_custom option contains a list of Doctrine relations that may be embedded. These relations will only be embedded if the embed parameter is present in the request. The behavior is consistent with the embed_relations option. Here is a valid configuration for our "Post" model:

        config:
          get:
            embed_relations_custom:                [ PostCategory ]

If the embed request parameter is given, this configuration will produce a feed like:

...
<Post>
  <Id>1</Id>
  <PostCategoryId>2</PostCategoryId>
  <CreatedBy>26</CreatedBy>
  <Title>Here the title of my post</Title>
  <Summary>Here the summary of my post</Summary>
  <Body>Here the body of my post</Body>
  <PostCategory>
    <Id>2</Id>
    <Name>Name of the category</Name>
    <Description>Description of the category</Description>
    <IsEnabled>1</IsEnabled>
  </PostCategory>
</Post>
...

embedded_relations_display

You may want to show only specific fields in the result. These fields can be configured with the embedded_relations_display configuration option:

        config:
          get:
            embedded_relations_display:
              PostCategory:
                - name
                - description

Note: this configuration option only has effect when the display option is set. When there is no embedded_relations_display option for a certain embed, all fields will be selected.

embedded_relations_hide

You may want to hide some fields from the embedded relations. For instance, you could want to hide the id field from the PostCategory model. This can be done using the embedded_relations_hide configuration option:

        config:
          get:
            embedded_relations_hide:
              PostCategory:                     [id]

global_additional_fields

In some case, you might want to embed some additional fields in the XML or JSON response. For instance, you might want to include the total number of posts, an average price, etc.

The global_additional_fields is helpful in such a situation. It contains an array of the fields that you want to add and, for each field, the generator will create a method dedicated to embed this field. Here is a possible configuration:

        config:
          get:
            global_additional_fields:                       [ TotalPosts ]

This will create an empty method, which has to be manually overridden in the generated module, in order to include the additional field of your choice:

public function embedGlobalAdditionalTotalPosts($params)
{
  $totalObjects = count($this->objects);
  $this->objects['NbObjects'] = $totalObjects;
}

hide

The hide option contains the list of the fields to hide in the XML or JSON feed. For example with the previously defined "Post" model, you can choose to hide its id by changing this parameter:

        config:
          get:
            hide:           [id]

This option has the priority over the "display" option, which means that if both options are used, only the fields that are not listed in the "hide" option will be rendered.

max_items

This directive allows to fix an absolute limit to the number of items in the response. This parameter has the priority over the pagination_page_size directive, and the possibly user's defined page_size parameter.

There is by default no limit. Setting this key to 0 will disable the limit.

object_additional_fields

The object_additional_fields contains the list of the additional fields that have to be embedded in each item of the response. For instance, if you want to add a field NbWords, which would give the number of words in the body of the post, use the following configuration:

        config:
          get:
            object_additional_fields:                       [ NbWords ]

This will create an empty method, which has to be manually overridden in the generated module, in order to include the additional field of your choice:

public function embedAdditionalNbWords($item, $params)
{
  $array = $this->objects[$item];
  $array['NbWords'] = str_word_count($array['body']);
  $this->objects[$item] = $array;
}

This option is useful for embedding a relation with only a few fields (see the embed_relations option):

embed the relation
use the object_additional_fields option to unset the non-desired fields.

The embedAdditionalXXX() methods should always have the following form (the generator generates this code as comments):

public function embedAdditionalXXXX($item, $params)
{
  $array = $this->objects[$item];

  // here go some manipulation of $array

  $this->objects[$item] = $array;
}

It's also possible to specify which method should be called to add the field:

config:
  get:
    object_additional_fields:
      is_deleted: additionalFieldIsDeleted

The specified function should have this signature:

function additionalFieldIsDeleted(array $object, array $parameters): mixed
{
  // $object contains the object as it currently is formatted
  // $parameters contains the request parameters from this API call
  // return the value you'd like field "is_deleted" to have
}

and the specified function will be called in this way:

// in formatObjects:
$this->objects[$key]['is_deleted'] = $this->additionalFieldIsDeleted($this->objects[$key], $parameters);

pagination_enabled

This option defines whether or not the pagination should be enabled. Defaults to false. If enabled, the service will allow a parameter "page" to be passed in the request. The request can then be of the form http://api.example.org/post/?page=3

pagination_custom_page_size

Set this option to true to allow the client to pass a page_size parameter. Else, the pagination will have a fixed size. If the pagination_enabled option is set to false, this option will have no effect.

pagination_page_size

This option defile the default page size of the pagination. If the pagination_enabled option is set to false, this option will have no effect.

sort_custom

Set this option to true to allow the client to pass a sort_by and a sort_order parameter in query string. Else, the client will not be able to sort the results.

sort_default

The sort_default option defines the default sort order. The format of this option is [column, asc|desc]. For example:

        config:
          get:
            sort_default:                       [ created_at, desc ]

filters

This option allows to override the default filtering behavior by setting some options. By default, the plugin allows to filter the results based on the model's fields. For each field, it is possible to pass a value in query string, which will be used to select the matching items.

For instance, you might want to get only the posts of a certain category using a category_id parameter in the request. If you want to allow the client to request the posts of several categories, you have to explicitly allow it, as it may create more complex (ie. resource-consuming and slow) requests. In that goal, the key multiple has to be set to true for this field name:

        config:
          get:
            filters:
              category_id:                  { multiple: true }

This will allow to call the webservice with several category_ids at once, with a request of the form http://api.example.org/post/?category_id=1,4,5

For the dates fields, you might want to tell the plugin which date format is accepted. For example:

        config:
          get:
            filters:
              created_at:                  { date_format: 'd-m-Y' }

To compare a certain field with a given value, you can add a compare directive to add filters to a certain field. These filters take the name fieldNameOperator, the following operators are supported:

less
lessEqual
greater
greaterEqual

Optionally, the compare filters take an alias name, which overrides the default operator names in the parameter.

For example, to add compare filters to the created_at field:

        config:
          get:
            filters:
              created_at:
                compare:
                  less: before # generates the created_atBefore filter
                  lessEqual: before # generates the created_atLessEqual filter
                  greater: after # generates the created_atAfter filter
                  # greaterEqual will not be generated

create

The create option lists several options specific to the "create" operation:

disable_validators

The disable_validators options contains the list of field names for which create validators should not be generated. For example:

        config:
          create:
            disable_validators:            [ created_at, updated_at ]

Other configuration variables

Some other configuration variables are not present in the default configuration file:

the actions_base_class parameter allows to change the name of the base action class which is extended by the module's action class. This permits to use your own action class, which may package several methods which you will want to use in several REST modules.

Events

As of version 0.9.1, the plugin uses events at several places, in order to improve to overload and extend the default behavior.

Here is a list of the supported event names :

sfDoctrineRestGenerator.filter_error_output: This event is launched in order to filter the error message and enable its customisation
sfDoctrineRestGenerator.filter_results: This event is launched in order to filter the query result array (add, remove or changes some keys of it). Note that you can achieve the same thing by using the object_additional_fields parameter in the generator.yml
sfDoctrineRestGenerator.filter_result: This event is launched in order to filter a single query result (add, remove or changes some keys of it). Note that you can achieve the same thing by using the object_additional_fields parameter in the generator.yml
sfDoctrineRestGenerator.get.pre: This event gets fired at the very beginning of a request (at the beginning of executeIndex())

Whishlist

If you use the plugin and want to help me improve it, you could consider picking one of the following topics:

possibility to nest the embed_relations parameter (and not limit it at one level only)
possibility to disable events notification / filtering (performance)
more serializers (BSON or RDF for example). Currently, the plugin only allows to serialize the resultsets as a XML, YAML or JSON feeds (see the chapter "Serialization"). Mobile clients, which require the most compact possible streams, would take benefit from a BSON serialization.
possibility to generate client libraries (sfDoctrineRestClientGenerator ?)
possibility to generate unit tests
possibility to generate API documentation
document authentication solutions
all the possible feedback!

Contribute to the plugin, ask for help

Please ask for help on how to use the plugin on symfony's users mailing list. You can also send me a mail directly : xavier@lacot.org.

License and credits

This plugin has been developed by Xavier Lacot and is licensed under the MIT license.

Changelog

trunk

Enabled the show route by default

version 0.9.4 - 2010-11-25

Allow to create or update related objects in the very same request: payload parsing in a recursive way, recursive validation of the payload array, recursive generation of the validators array. Thanks to dfeyer for his contributions on this topic
Added a basic "show" action (thanks dfeyer)
Improved the performance of the XML serializer
Added the capacity to serialize single objects (thanks dfeyer)
Added cleanupParameters() method to avoid code duplication (thanks dfeyer)
Added setFieldVisibility() method, in order to share code between index and show actions (thanks dfeyer)
Added configureFields() method, in order to share code between index and show actions (thanks dfeyer)
Fixed executeUpdate() so that the content is not a parameter of the request, but the content of the PUT request (thanks dfeyer)
Fix the setFieldVisibility() method, so that object additional fields don't get stripped if they're not listed in the 'get.display' option
Have the XML serializer return a PHP array (thanks dfeyer)
Added the formats_strict option
Improved the deserialization abstraction
Added a YAML serializer
Improved the documentation on these topics

version 0.9.3 - 2010-08-13

Added hide and embedded_relations_hide options (thanks to Pascal Borreli)
Made the fields filtering faster

version 0.9.2 - 2010-07-01

Added events (thanks to Matthew Penrice)
Added a default_format parameter (thanks to Matthew Penrice)
Added PUT support
Switched to json as default serializer (faster and less verbose)
Improved the documentation

version 0.9.1 - 2010-05-14

Fixed a typo in the previous release

version 0.9 - 2010-05-14

Added a JSON serializer.

version 0.8 - 2010-05-09

Initial public release. Features REST module generation with validation and a XML serializer.

recras / sf-doctrine-rest-generator-plugin

Maintainers

Details