27 Jan 2016, 09:30

Around the Web - January 2016 - Website obesity crisis, AngularJS & Postgres

Website Obesity

  • Website obesity crisis : transcript of a talk (video can be seen too from the link) about webperformance, bloated websites and how fat sites become for almost no real value for end users. Long but funny and instructive. It makes you think on how complex, fat and bloated the web is nowadays, in terms of tooling, architecture, code and content.

AngularJS

Postgres

  • The most advanced opensource database of the world, ie Postgres for short, was released as 9.5 version (FR / EN) ; it brings the long expected "UPSERT" features, Row level security and some big-data features (improved index, faster sorts, connection to Hadoop/Cassandra via FDW, etc)
  • In French a deeper view of the 9.5 release (part 1, part 2, part 3) to better understand what contains this release.

 

23 Dec 2015, 09:30

Around the Web - December 2015 - AngularJS & AMP

AngularJS

AMP

Google released AMP (Accelerated Mobile Page) which is a subset of HTML to have the minimal formatting for content but possibly with ads and analytics embeded. It aims to speed up the web as Google defined it :

Instant. Everywhere.

For many, reading on the mobile web is a slow, clunky and frustrating experience - but it doesn’t have to be that way. The Accelerated Mobile Pages (AMP) Project is an open source initiative that embodies the vision that publishers can create mobile optimized content once and have it load instantly everywhere.

  • Online advertising - current situation (in French) : A summary to have the whole context on online advertising, the decrease of ads revenues, the rise of ad blockers and the possible alternatives, such as AMP but not only.
  • AMP and Responsive Web Design  : a very interesting article which describe what AMP is and isn't ; in a few words:
    • AMP is more an answer to Facebook Instant's article and apple news
    • APM is strictly performance oriented but not SEO or accessibility oriented
    • It does not threat RWD even if it can show some ways of improvments
  • Why AMP is fast : the article details how in terms of techniques, code and platform, APM is fast. You can take some best practices from it to apply to your websites (when relevant)

26 Nov 2015, 23:59

Retour sur Codeurs en Seine 2015

Aujourd'hui se tenait l'édition 2015 de Codeurs en Seine ; petit résumé de la journée :

  • Test Drive Infrastructure avec Docker (slides ; code):
    • L'idée est d'appliqué les principes du TDD sur la partie infrastructure (versionning, tests, intégration continue, etc).
    • Différents niveaux de tests possibles
      • Tests unitaires : tests au niveau de la fabrication du container en lui même
      • Tests d'intégration : tests du bon fonctionnement des composants de l'infrastructure
      • Mais aussi tests d'acceptance, tests de sécurité, tests de performance, etc.
    • Développement de docker-unit qui permet de tester la bonne exécution d'un docker build en testant le résultat du Dockerfile à chaque étape.
    • Le projet est basé sur dockramp, il est encore jeune mais l'idée est intéressante. L'auteur du projet doit contacter l'équipe Docker pour voir s'il doit continuer ou si cette dernière a déjà des choses dans les cartons.
  • 5 facteurs clés pour l'auto-organisation : l'auteur revient sur l'organisation d'Agile France qui s'est fait dans un contexte particulier. Il en retient 5 clés :
    • Ownership : il a fallu dépasser sa conception "égocentrique" de l'événement pour en faire l'événement de la communauté pour fédérer tout le monde autour du projet
    • Dialogue : passage de la discussion (bruit++) au dialogue (signal++) ; seules les personnes ayant un intérêt sur un sujet donné ont le droit de donner leur avis et de travailler ensemble. Le bruit ambiant généré par les autres est supprimé en les "excluant" du sujet (ou du moins en ignorant leur "moi je pense que...")
    • Leadership ; en prologement du point précédent, plutôt que de chercher un consensus mou, que les personnes intéressées par le sujet prennent le leadership du sujet et avancent. A noter qu'il n'y a pas d'opposition entre auto-organisation et leadership, c'est le passage d'avis vers des intentions.
    • Artefacts : il s'agit de produire des choses visibles et de s'organiser autour de ces artefacts (réunions, etc)
    • Vision : il faut définir un cadre et communiquer autour de ce cadre.
    • En reprenant de bas en haut, un moyen de se rappeler du mot via le terme VALDO.
  • HTTP/2, les bonnes pratiques du web évoluent ; une présentation rapide de HTTP/1.1 et des apports de HTTP/2 avec les principaux apports (push server, multiplexage, priorité des ressources, compression des entêtes, flux binaire plutôt que textuel et https quasi requis de part le support de http/2 dans Chrome/Firefox) et impacts sur nos pratiques actuelles (plus besoin de domain sharding, de concaténation, de sprites CSS ou d'inlining). Par contre, les optimisations d'images, la compression, la minification et l'optimisation des fontes et la gestion du cache restent de mise.
  • Bidouillabilité à l'ère du numérique : Tristan Nitot qui nous parle
    • des débuts de l'ordinateur où ils étaient bidouillables vs les modèles de plus en plus fermés de nos jours
    • des débuts du web s'appuyant sur des formats ouverts vs des sdk "fermés" où en gros on doit demander l'autorisation de...
    • du cloud (dans le sens des offres SaaS) qui n'est rien d'autres que l'ordinateur de quelqu'un d'autre et sur lequel vous n'avez aucun contrôle, dont le code source n'est pas disponible et qui n'est pas bidouillable.
    • Dans le SaaS, le client est celui qui paye pour les données, pas celui qui les fournit (contre un service "gratuit")
    • Tristan Nitot propose le SIRCUS : Système d'Information Redonnant le Contrôle Aux UtilisateurS et ses 7 principes
      • Pas de publicité ciblée
      • Utiliser du matériel que l'on contrôle (Raspberry, CubieTruck, etc en auto hébergement ou à la rigueur chez un hébergeur)
      • Utiliser du logiciel libre (CozyCloud, Owncloud, YunoHost, etc)
      • Utiliser le chiffrement
      • Une UX à la hauteur
      • Interopérabilité
      • Une killer feature que les offres SaaS ne peuvent fournir
        • A ce sujet, Cozycloud réfléchit à croiser les données qui seraient centralisées dans l'instance (ex afficher les noms des contacts en lieu et place de leur numéros sur la facture téléphonique en croisant la facture avec les contacts)
    • Mes enseignements :
      • Même en étant relativement sensibilité à la gestion des données personnelles, cette conférence donne une claque et montre le chemin à parcourir et les enjeux.
      • Toutefois la question de l'auto-hébergement reste un problème ; comment demander et permettre à Mme Michu d'avoir ses données de façon sécurisée ?
      • Par ailleurs, si CozyCloud arrive à tout concentrer en un seul lieu et à croiser les données, quid en cas d'intrusion sur le systmèe (via un hacker ou un cambrioleur ou des forces de l'ordre à la rigueur ?). Qu'est-ce qui est mieux entre une analyse partielle mais permanente des données chez les GAFA vs un risque faible mais un impact très fort si mon instance CozyCloud par ex est récupérée par un tiers, celle-ci contenant nos données ?
      • Il y a peut être des choses intéressantes à faire dans un contexte CozyCloud + VRM.
  • Apache Drill, le SQL pour Hadoop et plus... :
    • Drill permet de manipuler en SQL tout types de données issues d'un cluster hadoop, d'une base de données SQL/NoSQL, de fichiers CSV, JSON,XML, etc. Il permet même au sein d'une même requête de requêter sur plusieurs sources de données.
    • C'est donc du "SQL on everything" avec une logique de schema à la volée en fonction de la requête. Cela peut être distribué notamment dans un cluster Hadoop.
    • A intégrer dans les outils rapidement...
  • AngularJS, the good, the bad and the transition to Angular 2 :
    • Un talk qui présente les bons et mauvais côtés d'Angular V1, une rapide présentation d'Angular V2 et comment migrer
    • Un livre sur Angular2 est en cours de rédaction ; celui sur Angular V1 est dispo => books.ninja-squad.com ; j'ai la V1 mais toujours pas lu ;-)
    • The Good
      • ngHint
      • eslint + plugin Angular
    • The Bad
      • $scope-soup
      • Eviter les conflits de scope parents/enfants via les "ControllerAs"
      • Performances
        • Utliser les bindOnce pour réduire le nombre de watchers
        • Ajouter des "track by" pour éviter de reconstruire le dom d'une vue sur l'autre si on manipule les mêmes objets
        • Utiliser judicieusement ngIf vs ngShow
          • ngIf : détruit le dom et le reconsruit
          • ngShow : cache le bloc mais si celui-ci contient des instructions qui sont calculées, alors elles le seront quand même
        • ng-model-options pour conditionner le déclenchement du cycle de digest/watcher
      • Fuites mémoire ; rajouter des $scope.on("$destroy" ...) pour faire le ménage
    • Angular2
      • Prévoir de l'écrire en ES2015 ou TypeScript
      • Tout est composant
      • Nouveu modèle de template
        • Web Components compatible
        • Web Workers compatible
      • angular-cli
    • Migration vers NG2 dans le cadre d'un projet NG1
      • Utiliser la syntaxe ControllerAs
      • Utiliser les directives pour initier une approche composant ; Angular 1.5 apportera un début de syntaxe "component"
      • Commencer à écrire en ES6
      • ngUpgrade permettra d'utiliser une syntaxe orientée composant et de les "downgrader" en syntaxe Angular 1.
  • Ionic, le framework mobile hybrid carrément addictif :
    • Application hybride = WebView embarqué dans une application native
    • Ionic = AngularJS + Cordova
    • Plugins ng-cordova pour accéder au matériel et fonctionnalités du périphérique
    • Ionic fournit un ensemble d'outil pour aider au développement et au déploiement des applications avec un modèle économique en cours de définition
    • solution manquant encore un peu de maturité ?

Au final, globalement une bonne journée avec une bonne organisation. La diversité des tracks (agile, technologies, web et Java) permettent de trouver son bonheur et d'avoir un programme à la carte. Une bonne formule en somme.

26 Nov 2014, 09:30

Around the Web - November 2014

DNS

HTTPS

MySQL

HTML5/CSS/Responsive Web Design

  • 7 CSS Units you may not know about : rem, vh, vm, vmin, vmax, ex and ch. I did not know the two latter. Some more examples for the vh/vm and vmin/vmax.
  • About rem and em more specially, if you want to move from a fixed approach (ie pixel one) to a more fluid/adaptive one (em/rem), you should read this article  and then this one which explain the issue with pixels and the new way to manage it. You can also use em/rem for positionning content ; em/rem are not only about text.
  • 5 obsolete features in HTML5: hgroups tag, pubdate and scope attributes, command and center elements. With the good way to implement them and/or some workaround if you still need them.
  • RWD adoption 2014 : top 100/1000/10.000 sites are evaluated - from to what extend is RWD implemented to mobile site vs RWD benchmarks in terms of performance.
  • 6 technologies that will change the web platform : asm.js, paralleljs, ECMAScript 6, web components, installable webapps, CSS Grid layout
  • The state of Web Animation 2014 : Between the post-Flash area and the Web Animation API to be implemented in all browsers, a review of current challenges and polyfill to bring animations into the browsers. Comments are also worth to read to get more resources.
  • If you are interested in a book about RWD, seems the latest book from A book apart may interested you : Responsible responsive web design (related review)

Browser

Web Performance

  • M6 Tech team made a review of their participation to Velocity conf (day 1, day 2, day 3), a web performance oriented conference. Even if their synthesis is in French, related slides and video are in English. You can also find the one of 2013 (day 1, day 2, day 3)

AngularJS

React (Facebook)

  • React through the ages : an interesting introduction (from origin to what's coming) about React, a JS library to build user interfaces.

19 Mar 2014, 09:30

Misc - 19/3

UI

  • Brick is a bundle of reusable UI components based on HTML5 and made by Mozilla ; you can read an introduction about it. Seems minimalist to some extend (limited in numbers even if growing) but also very powerful at a qualitative point of view.

DevOps

  • If you are interested in Chef or more widely about IT automation, you can read 3 articles in French on how to run Chef from scratch and cook your first recipes.

Postgres

  • PostgreSQL 9.3.3, 9.2.7, 9.1.12, 9.0.16 and 8.4.20 released! It contains fixes for multiple security issues, as well as several fixes for replication and data integrity issues. It also contains many improvements.
  • Barman seems a very powerful backup and recovery manager for Postgres. Develop by the 2nd Quadrant team which are in the main commiters on Postgres code.
  • Postgresql Studio : a web app to manage your postgres database in a Java container (tomcat, etc). Seems a modern alternative to phpPgAdmin.
  • Postgresql Exercies : to discover Postgres world and train yourself to build SQL queries.

Development

  • Some best practices (in French) about POST vs PUT to use them correctly in your API.
  • The AngularJS team wrote a blog post about what would be in AngularJS 2.0 ; seems very promising both on the improvements part but also what they aim to ease. It's also interesting to see that they inspire from ES6 which is the next version of Javascript (not yet implemented in your browser)
  • Opquast is reviewing their best practices around mobile development til end of March, with a public review. Then final recommendation will be released. If you don't know Opquast (for Open Quality Standards), they released a lot of checklists to asses/review your site.
  • The truth about multiple h1 tags in the HTML5 era : with the change of structure allowed with html5, it's now safe to implement multiple h1 tags.
  • Wonder when/why you should use Node.js for your next project : read this article which explains the paradigm behind Node.js and where it fits best for projects.

19 Feb 2014, 09:30

AngularJS Review - 19/2

No time to code this week, so some readings with a focus on AngularJS

See you maybe next week or in two weeks after Winter break ;-)

12 Feb 2014, 09:30

Sailing app - part 5 - Removing some dust and improving the databinding with bindOnce

It's more than one month we last talked about our sailing app. Let's take some time to refresh it.

Removing the dust

Refreshing our tooling chain

When we initiated our environment, we used npm to install yeoman, bower and grunt mainly.

Let's update first the global package we installed (ie the one we install with npm install -g <package>) :

npm update -g

npm will check against its registry if updates are available. If yes, it will install the updates.

Refreshing the library we use for our project

If you go to your root directory and then run npm update and depending on how strict you defined your version policy in your package.json file, you may or may not get updates.

Let's take some time to review this : npm applies the semver principle to define how it should behave when updating.

In the package.json file, we have the simplified version below :

{
  "name": "vmcollection",
  "version": "0.0.0",
  "dependencies": {},
  "devDependencies": {
    "grunt": "~0.4.1",
    "grunt-autoprefixer": "~0.4.0",
    "grunt-concurrent": "~0.4.1",
    "grunt-contrib-clean": "~0.5.0",
    [...]
  },
  "engines": {
    "node": ">=0.8.0"
  },
}

Regarding grunt, the "tilde" would mean that when updating, it should take a version reasonably close to 0.4.1. So it would be any version number as of 0.4.x but not 0.5.0 or higher. It could have been written as "0.4.x" or "~0.4". As you can notice, all the dependencies in package.json will use the tilde notation to allow you smooth upgrades. Indeed, if developers use the right semver syntax, you should only get patch releases.

Regarding node, It would require any version greater than 0.8.

So the idea here is that you can manage your dependencies as you wish. From a strict point of view like "0.4.0" and whatever happens you will stick to it. Or have smooth upgrades (with tilde notation) or have more "risky" upgrade strategy with up to the greater or equal than version X.

You can test it still works by running the command we saw before like grunt test, grunt server, grunt build.

No regression ? Perfect, we can go further.

Refreshing our frontend libs

Now our tooling chain is updated, there may be some new version of AngularJS or JQuery that were released in the meantime. As for the package.json file above, bower have its own bower.json file and using the same principles.

So just run bower update and you're done. You will get the latest 1.2.x version of AngularJS and the related dependencies.

Everything is still fine in your tests and so on ? So let's improve our code.

Improving our binding with bindOnce

If you have some immutable data you manipuate, bindOnce may interest you as it will reduce the number of $watch (ie models mutation/changes in your scope - remember that your scope make the link between your model and your view.) So in a loop, you will have a $watch item for each ng-* directive you have. In our case, with 1800+ lines and 5 items per line, we have 9000+ $watch initiated. Maybe we don't need that much as we do not change the value of the data. That's the purpose of bindOnce.

So let's install it :

bower install angular-bindonce --save

In app/index.html, load the bindonce.js file :

    <!-- build:js scripts/modules.js -->
    <script src="https://nicolas.steinmetz.fr/post/2014/02/12/bower_components/angular-resource/angular-resource.js"></script>
    <script src="https://nicolas.steinmetz.fr/post/2014/02/12/bower_components/angular-cookies/angular-cookies.js"></script>
    <script src="https://nicolas.steinmetz.fr/post/2014/02/12/bower_components/angular-sanitize/angular-sanitize.js"></script>
    <script src="https://nicolas.steinmetz.fr/post/2014/02/12/bower_components/angular-route/angular-route.js"></script>
    <script src="https://nicolas.steinmetz.fr/post/2014/02/12/bower_components/angular-bindonce/bindonce.js"></script>
    <!-- endbuild -->

In app/scripts/app.js, add the bindOnce as a module in your app :

angular.module('vmCollectionApp', [
  'ngCookies',
  'ngResource',
  'ngSanitize',
  'ngRoute',
  'pasvaz.bindonce'
])
[...]

Before updating our template, let's introduce ngBind ; up to now, we had in app/views/index.html:

  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter | orderBy:orderProp">
    <td>{{ item.model }}</td>
    <td>{{ item.boat_type }}</td>
    <td>{{ item.category }}</td>
    <td>{{ item.month }}</td>
    <td>{{ item.year }}</td>
  </tr>

As you can read, it's recommended to use the ngBind syntax over the one above to avoid some template compilation issues. So we now have :

  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter | orderBy:orderProp">
    <td ng-bind="item.model"></td>
    <td ng-bind="item.boat_type"></td>
    <td ng-bind="item.category"></td>
    <td ng-bind="item.month"></td>
    <td ng-bind="item.year"></td>
  </tr>

And so now you can more easily see where our 9000+ $watch are linked with.

So let's improve it with the bindOnce directives (bo-*):

  <tr class="boat" bindonce ng-repeat="item in issuelist | filter:basicboatfilter | orderBy:orderProp">
    <td bo-text="item.model"></td>
    <td bo-text="item.boat_type"></td>
    <td bo-text="item.category"></td>
    <td bo-text="item.month"></td>
    <td bo-text="item.year"></td>
  </tr>

So now, if all works well, we only have one single $watch at the ng-repeat level and no longer within the loop. Unfortunately, I don't know yet how to show it easily.

Last but not the least, at test level, don't forget to add in karma.conf.js, in the files section the reference to bindonce.js :

    // list of files / patterns to load in the browser
    files: [
      'app/bower_components/angular/angular.js',
      'app/bower_components/angular-mocks/angular-mocks.js',
      'app/bower_components/angular-resource/angular-resource.js',
      'app/bower_components/angular-cookies/angular-cookies.js',
      'app/bower_components/angular-sanitize/angular-sanitize.js',
      'app/bower_components/angular-route/angular-route.js',
      'app/bower_components/angular-bindonce/bindonce.js',
      'app/bower_components/jquery/jquery.js',
      'app/scripts/*.js',
      'app/scripts/**/*.js',
      'app/vendor/**/*.js',
      'test/mock/**/*.js',
      'test/spec/**/*.js'
    ],

There is nothing to do on build grunt task as the bower_components directory in which bindOnce is installed is already managed.

Final good news is that bindOnce will be integrated in AngularJS 1.3 !

18 Dec 2013, 09:30

Sailing app - part 3 - AngularJS does not only fetch content, it can transform too

In part 1, I cheated to provide a JSON file AngularJS could directly interact with but as I told you, it came from an MS Excel file I converted into a CSV one. We can in fact avoid this XLS > CSV > JSON conversion and make the CSV to JSON conversion on the fly when application is loaded.

For today, we'll need :

  • The CSV version of the file : app/data/index-essais-215.csv
  • A library to transform the CSV file to JSON ; I quickly chose JQuery-CSV as it made the job even if it add JQuery as a dependancy (which is already used for the project somewhere as it is defined in bower.json file). Files will be stored in app/vendor/jquery-csv.

As all pre-requisites are satisfied, let's update our code :

In app/index.html, add the jquery-csv at the end, before </body> :

    <!-- CSV parser and JQuery as pre-requisites -->
    <script type="text/javascript" src="https://nicolas.steinmetz.fr/post/2013/12/18/vendor/jquery-csv/jquery.csv-0.71.min.js"></script>

In app/scripts/controller/index.js :

  • We'll change the $http.get() shortcut for a more complex http call. It will introduce the "transformResponse" method and the cache property.
  • As previously, this is made asynchronously
  • Once the file is fetched, we go into the transformResponse section in which we will transform the CSV file into a JSON array.
angular.module('vmCollectionApp')
  .controller('IndexCtrl', ['$scope', '$http', function ($scope, $http) {
    $http({
      method: 'GET',
      url: "./data/index-essais-215.csv",
      transformResponse: function(issuelist) {
        // Transform CSV file into a JSON object
        var json = $.csv.toObjects(issuelist);
        return json;
      },
      cache: true,
    })
    .success(function(issuelist, status) {
        $scope.issuelist = issuelist;
    })
    .error(function(data, status) {
      $scope.issuelist = issuelist || "Request failed";
    });

    $scope.orderProp = 'model';   
  }]); 

Almost perfect, still our test to update , ie in test/spec/controllers/index.js :

  • We need to change only the beforeEach section in which the file was fetched and fake the content of the CSV file
  // Initialize the controller and a mock scope
  beforeEach(inject(function (_$httpBackend_, $controller, $rootScope) {
    // Define the expect answer from the http request and content of the json file supposed to be fetched
    var issuelist = 'model,boat_type,category,issue,month,year\n \
      420,dériveur,présentation,11,novembre,1996 \n \
      420,dériveur,une occasion mise à nu,11,novembre,1996';

    // Initiate httpbackend service to get the json file in asynchronous mode and to fake the $http service
    $httpBackend = _$httpBackend_;
    $httpBackend.expectGET('./data/index-essais-215.csv').respond(issuelist);
   
    scope = $rootScope.$new();
    IndexCtrl = $controller('IndexCtrl', {
      $scope: scope
    });
  }));

If you run grunt test at this step, it... fails !

Indeed, you need to make karma (the test runner) aware of jquery and jquery-csv. For that, just edit karma.conf.js in the project root directory. You need to update the files section with the jquery declaration and the app/vendor "global" one :

    // list of files / patterns to load in the browser
    files: [
      'app/bower_components/angular/angular.js',
      'app/bower_components/angular-mocks/angular-mocks.js',
      'app/bower_components/angular-resource/angular-resource.js',
      'app/bower_components/angular-cookies/angular-cookies.js',
      'app/bower_components/angular-sanitize/angular-sanitize.js',
      'app/bower_components/angular-route/angular-route.js',
      'app/bower_components/jquery/jquery.js',
      'app/scripts/*.js',
      'app/scripts/**/*.js',
      'app/vendor/**/*.js',
      'test/mock/**/*.js',
      'test/spec/**/*.js'
    ],

And we're done. grunt test should pass now and we have the same features as at the end of part 2.

Idea of this part was to show you that you don't strictly need JSON objects/array with AngularJS. You only need a ressource you can fetch and that you can "manipulate". Some other use cases could be the consumption of XML, RSS feeds, etc.

Of course, you can grab the code and see the results.

11 Dec 2013, 09:30

Sailing app - part 2 - Two way data binding in AngularJS

At the end of part 1, we just have the full boat list. If you would like to see only the issues from 1996 or the issues which mentionned the "470", you need some more things. It's also a quick way to introduce the two way data binding of Angular :

Data-binding in Angular web apps is the automatic synchronization of data between the model and view components. The way that Angular implements data-binding lets you treat the model as the single-source-of-truth in your application. The view is a projection of the model at all times. When the model changes, the view reflects the change, and vice versa.

So what a better example of this data binding with filtering/sorting data ?

Basic filtering

To quickly illustrate this notion, let's just add a input field in our html page. Issue list will be filtered dynamically according to the value of the field (whatever the column it matches) : If I enter 1996, I'll have all the issues from year 1996 ; If I fille 420, I'll see issues about the 420 boat, etc.

In app/views/index.html, just add :

  • A input field in which you will enter the value you look for ; it will be linked to the scope and "binded" via the ng-model. Thus, the value of the input field is linked to our controller.
  • On the ng-repeat line, just add the filter to update the value of the view based on the value of the input field.
<p>We have {{ issuelist.length }} entries</p>
<form>
  <p>Search: <input ng-model="basicboatfilter"></p>
</form>
<table>
  <thead>
    <td>Model</td>
    <td>Boat type</td>
    <td>Category</td>
    <td>Month</td>
    <td>Year</td>
  </thead>
  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter">
    <td>{{ item.model }}</td>
    <td>{{ item.boat_type }}</td>
    <td>{{ item.category }}</td>
    <td>{{ item.month }}</td>
    <td>{{ item.year }}</td>
  </tr>
</table>

And voilà ! But not, a test is missing !

Another feature of Karma is to provide end to end testing in addition to unit testing as we saw previously. It will simulate actions on the application side as a human will do. The scenario will be :

  • First, open the "index" page, ie the one we use to display the boat list ; this is done with the browser().navigateTo().
  • Once page is loaded, check you have at least 3 entries
  • Input "Belem" in the input field and check you have only 1 entry
  • Input "Dériveur" in the input field and check you have 195 entries

Create test/e2e/scenario.js :

'use strict';

/* http://docs.angularjs.org/guide/dev_guide.e2e-testing */

describe('VMCollection App', function() {

  describe('Boat list view', function() {

    beforeEach(function() {
      browser().navigateTo('/#/index');
    });

    it('should filter the boat list as user types into the search box', function() {
      expect(repeater('tr.boat').count()).toBeGreaterThan(3);

      input('basicboatfilter').enter('Belem');
      expect(repeater('tr.boat').count()).toBe(1);

      input('basicboatfilter').enter('dériveur');
      expect(repeater('tr.boat').count()).toBe(195);
    });
  });
});

With some configuration in Gruntfile.js and karma-e2e.conf.js I'll not describe here, we have :

grunt karma:e2e
Running "karma:e2e" (karma) task
WARN [config]: urlRoot normalized to "/_karma_/"
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/_karma_/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket BOJKfaU48e98r2mcJEWw
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 1 of 1 SUCCESS (6.431 secs / 5.045 secs)

Done, without errors.

And voilà !

Now we have a fully working "basic" filter.

Model based filtering

If we want some filters applied to a given column of the table, we need something more tied with the model of our data.

With what we saw before, we need :

  1. a variable we'll use to make our app aware of the order we want ; let's call it ordreProp. It will be a drop down list based on a select tag and will contain a few filter options. Value of the options will match the name of the fields of our model.
  2. $scope being aware of it to update the view and the model accordingl ; we'll do it with ng-model
  3. We need to make our loop based on ng-repeat aware of our sort criteria ; we'll add a orderBy property
  4. We need to have a default order for the sort when none is selected

We'll cover the three first points in on template side, in app/views/index.html :

<p>We have {{ issuelist.length }} entries</p>
<form>
  <p>Search: <input ng-model="basicboatfilter"></p>
  <p>Sort by:
    <select ng-model="orderProp">
      <option value="model">Model</option>
      <option value="boat_type">Boat type</option>
      <option value="year">Year</option>
    </select>
  </p>
</form>
<table>
  <thead>
    <td>Model</td>
    <td>Boat type</td>
    <td>Category</td>
    <td>Month</td>
    <td>Year</td>
  </thead>
  <tr class="boat" ng-repeat="item in issuelist | filter:basicboatfilter | orderBy:orderProp">
    <td>{{ item.model }}</td>
    <td>{{ item.boat_type }}</td>
    <td>{{ item.category }}</td>
    <td>{{ item.month }}</td>
    <td>{{ item.year }}</td>
  </tr>
</table>

Point 4 is managed on controller side by defining a default value for orderProp in app/scripts/controllers/index.js :

'use strict';

angular.module('vmCollectionApp')
  .controller('IndexCtrl', ['$scope', '$http', function ($scope, $http) {
    $http.get('./data/index-essais-215.json')
      .success(function(data) {
        $scope.issuelist = data;
      })
      .error(function(data) {
        $scope.issuelist = data || "Request failed";
      });
 
    $scope.orderProp = 'model';
  }]);

We can improve our unit test by testing that ordreProp is well defined and contains the expected value and add in test/specs/controllers/index.js :

  it('list should be sorted by default against model', function(){
    expect(scope.orderProp).toBeDefined();
    expect(scope.orderProp).toBe('model');
  });

But also our end to end testing by adding in test/e2e/scenario.js :

    it('should be possible to sort via the dropdown list', function(){
      select('orderProp').option('Boat type');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
      select('orderProp').option('Year');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
      select('orderProp').option('Model');
      expect(repeater('tr.boat').count()).toBeGreaterThan(1);
    });

And to validate it and you will now see that both unit and end-to-end tests are automatically run :

grunt test

[...]

Running "karma:unit" (karma) task
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/
INFO [launcher]: Starting browser Chrome
WARN [watcher]: Pattern "/Users/nsteinmetz/Documents/Projets/angular/vmcollection/test/mock/**/*.js" does not match any file.
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket 2bAqI7hO57KA0HRQYdmb
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 3 of 3 SUCCESS (3.286 secs / 0.153 secs)

Done, without errors.

Running "karma:e2e" (karma) task
WARN [config]: urlRoot normalized to "/_karma_/"
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/_karma_/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket 3v7PDSR0rlOVe7KPZz4n
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 2 of 2 SUCCESS (19.25 secs / 17.477 secs)

Done, without errors.

Of course, check also in your browser that it works well ;-)

By the way, did you also notice you can combine the search and the sort crieterion ?

As for last time, you can see the results and get the code.

Synthesis

What we saw with this step 2 :

  • Data binding in AngularJS with basic filtering in template only and advanced one with filters
  • End to end testing with Karma

With part 1 and 2, you can see that with AngularJS mainly and with the help of yeoman/grunt/bower, you can quickly have a skeleton for your application with all batteries included (dependencies, unit test, e2e tests, etc). Focusing on AngularJS, we have some native features with a few lines of code.

Need to find now what would be in step 3... Don't know yet if it would be more on backend side or frontend side...

04 Dec 2013, 09:30

Sailing App - part 1 - Intro to Yeoman, Grunt, Bower and AngularJS

I'm fan of sailing and the magasine "Voile Magasine" provides some doc / pdf / excel file with the index of all their issues and the boat they tested. Purpose of our app wil be to render the excel file in a user friendly manner to allow user to filter depending on different criterion.

As you can see, the file has a few columns :

  • Boat model
  • Boat type
  • Test type
  • Issue Number
  • Month
  • Year

So you quickly can imagine on what we'll do in our app.

So let's start !

Disclaimer : I'll discover the tools on my way to build the app ; if you have best practices & so, please share and improve my writings :-)

Installing Yeoman / Grunt / Bower

So in this first part, the point is to define how we will work and get the " develoment tooling" chain. I decided to use Yeoman, which stands for a bunch of tools to allow the development of modern web apps. It includes :

  • yo - the scaffolding tool from Yeoman
  • bower - the package management tool
  • grunt - the build tool

First, it requires you to have a working instance of Node.js installted on your computer. I used the 0.10.22 version of Node. I'll not cover the node installation, on my mac, I just made a :

brew install node

Check that npm (the Node Package Manager) is installed and available.

So to install Yeoman, this should be sufficient :

npm -g install yo

If it clainms about bower version, just do first :

nrpm install -g bower grunt-cli

The -g installs the software globally for your node environment and not in the current directoty in a node_modules directory.

And we're done for yeoman :-)

Let's initiate our project ; we want to build it with AngularJS, the MVC Javascript framework, and fortunately Yeoman has a genreator for AngularJS, meaning that Yeoman provides some integration with AngularJS.

Let's install the generator :

npm install -g generator-karma
npm install -g generator-angular

The Karma generator was required by angular's one ; Karma is a tester runner in Javascript.

Project initalisation

So now we can initiate our projet, we'll call "vm-collection" :

mkdir vm-collection && cd $_
yo angular vm-collection --minsafe
[?] Would you like to include Twitter Bootstrap? (Y/n)Y
[?] Would you like to use the SCSS version of Twitter Bootstrap with the Compass CSS Authoring Framework? Yesn) Y
[?] Which modules would you like to include? (Press <space> to select)
angular-resource.js
angular-cookies.js
angular-sanitize.js
> angular-route.js

Select all modules (especially route at least) so far ; and npm will work for a few minutes to retrive all the required stuff. The --minsafe argument is to make you app ready for minification when it will be about to be deployed.

You did not see but the bower tool was used to retrived some libs. Bower is a package manager for frontend libs like jquery and others. It's well explained on the Yeoman page decidated to Bower. Libraries installed with Bower will be in the app/bower_components folder and their list will be in the bower.json file in your project root folder.

For example, if we want to use the Modernizr lib for HTML5/CSS3 detection in our project, just type :

bower install modernizr --save

The --save is useful to store the information you installed Modernizr in bower.json. When you share your project with others, you no longer need to save your lib in your source control system oher whatever. Just provide your bower.json file with all required libs (and versions if necessary) and your peers will have just to type bower install to fetch all the required libs.

At this step, if you run grunt server, it will open your default browser and you will see the demo page of the angular generator. The server has some watch/auto-reload mechanism which will reload the latest files when changed/saved and reload your current page. grunt test will launch the tests with karma but we'll see this later.

First Angular controller / route / template and test

I will not introduce AngularJS by itself and would recommend you to go through the tutorial at least to understand what will be done. Nevertheless, I'll give some explaination on what I'll do.

First of all:

  • I transformed the xls file from Voile Magasine into a CSV one I put in app/data/index-essais-215.csv
  • I transformed it then into a JSON document as even if I found a CSV to JSON on the fly library with  jquery-csv ; I had some issue in tests. So I converted it once for all in JSON ; File is app/data/index-essais-215.json 
  • For this tutorial, I have no admin interface nor backend storage to keep it as simple as possible.

Let's create our first "route" in Angular, which would results in :

  • a added route in app/scripts/app.js,
  • a new controller app/scripts/controllers/index.js
  • and a new view in app/views/index.html.
  • The controller being added to app/index.html

Without the yeoman generator, we should have done it manually, which is time saving.

yo angular:route index --minsafe
invoke angular:controller:/usr/local/lib/node_modules/generator-angular/route/index.js
create app/scripts/controllers/index.js
create test/spec/controllers/index.js
invoke angular:view:/usr/local/lib/node_modules/generator-angular/route/index.js
create app/views/index.html

As we saw previously, we added the --minsafe argument to be minification ready when necessary.

In app/scripts/controllers/index.js

'use strict';

angular.module('vmCollectionApp')
.controller('IndexCtrl', ['$scope', '$http', function ($scope, $http) {
$http.get('/data/index-essais-215.json')
.success(function(data) {
$scope.issuelist = data;
})
}]);

What we have and added :

  • The definition of the IndexCtrl controller as part of the vmcollectionApp.
  • The use of the $http service of AngularJS to access some resources through HTTP. We'll use it to access our json file.
  • Once the file is parsed, content is stored in a "issuelist" variable which we link to $scope to make it accessible from the template.

In app/views/index.html ; let's start with a basic rendering of our JSON file

<p>We have {{ issuelist.length }} entries</p>
<table>
<thead>
<td>Model</td>
<td>Boat type</td>
<td>Category</td>
<td>Month</td>
<td>Year</td>
</thead>
<tr ng-repeat="item in issuelist">
<td>{{ item.model }}</td>
<td>{{ item.boat_type }}</td>
<td>{{ item.category }}</td>
<td>{{ item.month }}</td>
<td>{{ item.year }}</td>
</tr>
</table>

What we have and added :

  • First line allow us to know how many entries do we have in the issuelist variable.
  • Then we build a table and thus use the ng-repeat directive to populate the lines of the table.
  • As we built an object, for each line, we access the value by its property name ; property names are defined in the first line of the json file.

In your console, run grunt server which will open a new browser ; then go to http://127.0.0.1:9000/#/index to see the basic rendering we made. The "/index" is not defined by magic, routing system is defined in app/scripts/app.js, where for an url the controller and the template to be used are defined.

You should see something like :

vmcollection_app_basic_rendering_table.png

We could say we are finished but we're not as there is not tests so far. Let'd do the related test !

Related unit is in test/spec/controllers/index.js :

'use strict';

describe('Controller: IndexCtrl', function () {

  // load the controller's module
  beforeEach(module('vmCollectionApp'));

  var IndexCtrl, $httpBackend,
    scope;

  // Initialize the controller and a mock scope
  beforeEach(inject(function (_$httpBackend_, $controller, $rootScope) {
    // Define the expect answer from the http request and content of the json file supposed to be fetched
    var issuelist = [{
      "model": "420",
      "boat_type": "dériveur",
      "category": "présentation",
      "issue": "11",
      "month": "novembre",
      "year": "1996"
      }];
    // Initiate httpbackend to fake the $http service
    $httpBackend = _$httpBackend_;
    $httpBackend.expectGET('/data/index-essais-215.json').respond(issuelist);
   
    scope = $rootScope.$new();
    IndexCtrl = $controller('IndexCtrl', {
      $scope: scope
    });
  }));

  it('issuelist should be non empty', function () {
    // JSON file is not yet retrieved so variable should be still undefined
    expect(scope.issuelist).toBeUndefined();
    // Get JSON data
    $httpBackend.flush();
// JSON file is retrieved so issuelist should be defined and have one entry ; cf issuelist content above, in the beforeEach section
expect(scope.issuelist.length).toBe(1);
// Go deeper, let's check properties
expect(scope.issuelist[0].model).toBeTruthy();
expect(scope.issuelist[0].boat_type).toBeTruthy();
expect(scope.issuelist[0].category).toBeTruthy();
expect(scope.issuelist[0].issue).toBeTruthy();
expect(scope.issuelist[0].month).toBeTruthy();
expect(scope.issuelist[0].year).toBeTruthy();
  });
});

What we did :

  • First, tests follow the Jasmine syntax.
  • the beforeEach section allow us to provide some fake content with the simulation of the $http service and providing a dummy content for our json file.
  • the test by itself will check if the issuelist is not defined before the test (it should not) ; then simulate the call to the JSON file and then will do what is defined in the controller and check that there is one entry.

And the answer when you run  grunt test :

[...]
Running "karma:unit" (karma) task
INFO [karma]: Karma v0.10.7 server started at http://localhost:8080/
INFO [launcher]: Starting browser Chrome
WARN [watcher]: Pattern "/Users/nsteinmetz/Documents/Projets/angular/vmcollection/test/mock/**/*.js" does not match any file.
INFO [Chrome 31.0.1650 (Mac OS X 10.7.5)]: Connected on socket I2qIcTpftG6bH-Z0UtQZ
Chrome 31.0.1650 (Mac OS X 10.7.5): Executed 2 of 2 SUCCESS (0.473 secs / 0.052 secs)

Let's stop here for this week with all what we did so far :

  • We installed our workflow chain with Yeoman, Grunt and Bower
  • We initiated our project and use the AngularJS generator from Yeoman to scaffold our application
  • We create a first full and complete Angular controller / route / view and unit test

See you next week (hopefully) !

[edit] : You can see the results and get the code.