11 Dec 2013, 09:30

Sailing app - part 2 - Two way data binding in AngularJS

At the end of part 1, we just have the full boat list. If you would like to see only the issues from 1996 or the issues which mentionned the "470", you need some more things. It's also a quick way to introduce the two way data binding of Angular :

Data-binding in Angular web apps is the automatic synchronization of data between the model and view components. The way that Angular implements data-binding lets you treat the model as the single-source-of-truth in your application. The view is a projection of the model at all times. When the model changes, the view reflects the change, and vice versa.

So what a better example of this data binding with filtering/sorting data ?

Basic filtering

To quickly illustrate this notion, let's just add a input field in our html page. Issue list will be filtered dynamically according to the value of the field (whatever the column it matches) : If I enter 1996, I'll have all the issues from year 1996 ; If I fille 420, I'll see issues about the 420 boat, etc.

In app/views/index.html, just add :

  • A input field in which you will enter the value you look for ; it will be linked to the scope and "binded" via the ng-model. Thus, the value of the input field is linked to our controller.
  • On the ng-repeat line, just add the filter to update the value of the view based on the value of the input field.
<p>We have {{ issuelist.length }} entries</p>
<form>
  <p>Search: <input ng-model="basicboatfilter"></p>
</form>
<table>
  <thead>
    <td>Model</td>
    <td>Boat type</td>
    <td>Category</td>
    <td>Month</td>
    <td>Year</td>
  </thead>
  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter">
    <td>{{ item.model }}</td>
    <td>{{ item.boat_type }}</td>
    <td>{{ item.category }}</td>
    <td>{{ item.month }}</td>
    <td>{{ item.year }}</td>
  </tr>
</table>

And voilà ! But not, a test is missing !

Another feature of Karma is to provide end to end testing in addition to unit testing as we saw previously. It will simulate actions on the application side as a human will do. The scenario will be :

  • First, open the "index" page, ie the one we use to display the boat list ; this is done with the browser().navigateTo().
  • Once page is loaded, check you have at least 3 entries
  • Input "Belem" in the input field and check you have only 1 entry
  • Input "Dériveur" in the input field and check you have 195 entries

Create test/e2e/scenario.js :

'use strict';

/* http://docs.angularjs.org/guide/dev_guide.e2e-testing */

describe('VMCollection App', function() {

  describe('Boat list view', function() {

    beforeEach(function() {
      browser().navigateTo('/#/index');
    });

    it('should filter the boat list as user types into the search box', function() {
      expect(repeater('tr.boat').count()).toBeGreaterThan(3);

      input('basicboatfilter').enter('Belem');
      expect(repeater('tr.boat').count()).toBe(1);

      input('basicboatfilter').enter('dériveur');
      expect(repeater('tr.boat').count()).toBe(195);
    });
  });
});

With some configuration in Gruntfile.js and karma-e2e.conf.js I'll not describe here, we have :

grunt karma:e2e
Running "karma:e2e" (karma) task
WARN [config]: urlRoot normalized to "/_karma_/"
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/_karma_/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket BOJKfaU48e98r2mcJEWw
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 1 of 1 SUCCESS (6.431 secs / 5.045 secs)

Done, without errors.

And voilà !

Now we have a fully working "basic" filter.

Model based filtering

If we want some filters applied to a given column of the table, we need something more tied with the model of our data.

With what we saw before, we need :

  1. a variable we'll use to make our app aware of the order we want ; let's call it ordreProp. It will be a drop down list based on a select tag and will contain a few filter options. Value of the options will match the name of the fields of our model.
  2. $scope being aware of it to update the view and the model accordingl ; we'll do it with ng-model
  3. We need to make our loop based on ng-repeat aware of our sort criteria ; we'll add a orderBy property
  4. We need to have a default order for the sort when none is selected

We'll cover the three first points in on template side, in app/views/index.html :

<p>We have {{ issuelist.length }} entries</p>
<form>
  <p>Search: <input ng-model="basicboatfilter"></p>
  <p>Sort by:
    <select ng-model="orderProp">
      <option value="model">Model</option>
      <option value="boat_type">Boat type</option>
      <option value="year">Year</option>
    </select>
  </p>
</form>
<table>
  <thead>
    <td>Model</td>
    <td>Boat type</td>
    <td>Category</td>
    <td>Month</td>
    <td>Year</td>
  </thead>
  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter | orderBy:orderProp">
    <td>{{ item.model }}</td>
    <td>{{ item.boat_type }}</td>
    <td>{{ item.category }}</td>
    <td>{{ item.month }}</td>
    <td>{{ item.year }}</td>
  </tr>
</table>

Point 4 is managed on controller side by defining a default value for orderProp in app/scripts/controllers/index.js :

'use strict';

angular.module('vmCollectionApp')
  .controller('IndexCtrl', ['$scope', '$http', function ($scope, $http) {
    $http.get('./data/index-essais-215.json')
      .success(function(data) {
        $scope.issuelist = data;
      })
      .error(function(data) {
        $scope.issuelist = data || "Request failed";
      });
 
    $scope.orderProp = 'model';
  }]);

We can improve our unit test by testing that ordreProp is well defined and contains the expected value and add in test/specs/controllers/index.js :

  it('list should be sorted by default against model', function(){
    expect(scope.orderProp).toBeDefined();
    expect(scope.orderProp).toBe('model');
  });

But also our end to end testing by adding in test/e2e/scenario.js :

    it('should be possible to sort via the dropdown list', function(){
      select('orderProp').option('Boat type');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
      select('orderProp').option('Year');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
      select('orderProp').option('Model');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
    });

And to validate it and you will now see that both unit and end-to-end tests are automatically run :

grunt test

[...]

Running "karma:unit" (karma) task
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/
INFO [launcher]: Starting browser Chrome
WARN [watcher]: Pattern "/Users/nsteinmetz/Documents/Projets/angular/vmcollection/test/mock/**/*.js" does not match any file.
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket 2bAqI7hO57KA0HRQYdmb
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 3 of 3 SUCCESS (3.286 secs / 0.153 secs)

Done, without errors.

Running "karma:e2e" (karma) task
WARN [config]: urlRoot normalized to "/_karma_/"
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/_karma_/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket 3v7PDSR0rlOVe7KPZz4n
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 2 of 2 SUCCESS (19.25 secs / 17.477 secs)

Done, without errors.

Of course, check also in your browser that it works well ;-)

By the way, did you also notice you can combine the search and the sort crieterion ?

As for last time, you can see the results and get the code.

Synthesis

What we saw with this step 2 :

  • Data binding in AngularJS with basic filtering in template only and advanced one with filters
  • End to end testing with Karma

With part 1 and 2, you can see that with AngularJS mainly and with the help of yeoman/grunt/bower, you can quickly have a skeleton for your application with all batteries included (dependencies, unit test, e2e tests, etc). Focusing on AngularJS, we have some native features with a few lines of code.

Need to find now what would be in step 3... Don't know yet if it would be more on backend side or frontend side...

09 Oct 2013, 09:30

Web Roundup - 9/10

Responsive web design

  • Viewport resizer, which defines itself as follow : Viewport resizer is a browser-based tool to test any website’s responsiveness. Just save the bookmarklet, go to the page you want to test, click on your created bookmarklet and check all kinds of screen resolutions of the page
  • RWD Bookmarklet : a bookmarklet which allow to see you to what extend your site is responsive with some features like making a keyboard appear, etc.

User Experience

Javascript

HTML/CSS

  • Clipping & Masking in CSS or how to use CSS to apply some "filters" on your images.
  • CSS @supports (CSS & JS) to detect browser support for a given (CSS) style directive. It means you will structure your code based on feature detection. Not fully convinced but worth to know it exists.

19 Feb 2013, 21:21

SQL Filter bypass

A series of (interesting) articles on SQL injection on how to bypass filters :

$Even if most of the cases are available for MySQL and in a PHP context, you may be interesting to have a look at some examples to rethink about the way you do your sql queries based on what you get from your app or the user (via forms or manual url guessing)

11 Dec 2012, 23:06

Postgres Array type

Beyond traditionnal data types (integer, text, boolean, etc), Postgres support some other interesting types (network, geographical, etc) and one I discovered a few weeks ago : arrays.

Idea is simple : instead of storing a single value in a colum, you can store several ones. Yes several ! You don't need to set up a "one to many relation" or "many to many relation" with others tables.

Simple example : let's imagine a blog application with posts.

Each post has :

  • A unique ID
  • A title
  • A content
  • Tags

Tags are generally managed via a "many to many relation" and thus would require at least two tables, nor to say three. (1 for post, 1 for tag and 1 for the post/tag relation)

With arrays in Postgres, you only need a single table :

create table post (
id         serial unique
title      text,
content    text,
tag        text[]
)

Any content type (at least traditional ones like text, integer, etc) can be an array, juste add "[]" after.

Back to my example ; let's create some content

INSERT INTO post(title, content, tag) VALUES ('Blog Post 1', 'Some Content for Post 1', '{"test", "post-1"}')
INSERT INTO post(title, content, tag) VALUES ('Blog Post 2', 'Some Content for Post 2', '{"test", "post-2", "another_test"}')
INSERT INTO post(title, content, tag) VALUES ('Blog Post 3', 'Some Content for Post 3', '{"test", "post-3", "a third test", "really !"}');

You can see that for each entry, the number of tags is not the same to illustrate that it can adapt to your needs if you did not define constraints.

Now, let's imagine you want to retrive post depending on tag value, quite easy :

# Select all posts with tags "test" and "post-1" (ie the first one)
select * from post where tag @> ARRAY['test', 'post-1']
# Select all post with tags "post-2" OR "post-3" (ie two last ones)
select * from post where tag && ARRAY['post-2', 'post-3']

For traditionnal schema, you may be used to use the "IN" statement for implementing the "OR" sample above, like :

select * from post where id in (1,2)

But for the "AND", a traditionnal approach would be  :

  1. Do a 1st query on post, filtering on the one having the tag "test"
  2. Do a 2nd query on the resultset from previous and filtering on "post-1"

If number of filters and join are known from beginning, it can be managed. If number of filters is dynamic (let's say user can filter on 1,2,3,...,n tags, you have to manage it on the fly.

With arrays in postgres, I find quite interesting to find a pure and simple SQL solution to manage both "And" and "Or" filtering on values and avoid also multiple tables or (complex) join queries. For dynamic queries (ie user will use from 1 to n tags, it would be a single function to manage it)

If you are interested in such content, I may speak later about hstore, the key/value system in Postgres.

PS : for those who know MongoDB, it can remain you the $all operatoir.

PS2 : You can also have a multiple dimension arrays with "text[][]" So for example, if you have a CRM database and want to keep the last monthly invoices for a customer for the last 3 years, you could define it this way : decimal[3][12]