Friday, December 11, 2015

Converting Nested Set Data to Tree Object

In a project at work we decided to use the nested set model. It fit very well with our needs. We set up the database schema and also created some procedures to help with inserting and deleting nodes. The next step is now reading the data from a client.  However we want to read the data and make sure the objects are represented in a tree structure.  So how do we take tabular data and convert it into a tree?  Easy, recursion.  We can sort the data by the left value, which should give us the ability to step through using a depth-first approach.

I'm not going to go into what the nested set model is, I'm assuming that since you are here you know something about it.  If not then I recommend the de facto blog here: http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/

Data Structures

First step is we need to define a data structure for our nodes.  It will contain the id, name, and list of child nodes.  You'll notice I set the properties for left and right to internal.  This is because there is no value giving users access to these properties so they should only be accessible within the project.


public class Node
{
 private double category_id = 0;
 private string name = "";
 private int left;
 private int right;
 private List<Node> nodes = new List<Node>();

 public Node()
 {
 }

 public Node(double category_id, string name, int left, int right)
 {
  this.CategoryId = category_id;
  this.Name = name;
  this.Left = left;
  this.Right = right;
 }

 public double CategoryId
 {
  get { return category_id; }
  set { category_id = value; }
 }

 public string Name
 {
  get { return name; }
  set { name = value; }
 }

 internal int Left
 {
  get { return left; }
  set { left = value; }
 }

 internal int Right
 {
  get { return right; }
  set { right = value; }
 }

 public List<Node> Nodes
 {
  get { return nodes; }
  set { nodes = value; }
 }
}

Okay this looks good.

Public Method

Next we can create the public method to open a connection to the database, load the data into a list, then pass that list off to a recursive function called "BuildNodes()" that will generate the node hierarchy, which we will then return.  Notice we are sorting the results the by left value in the SQL command.


public Node GetNodes()
{
 Node root = null;

 try
 {
  List<Node> nodes = new List<Node>();

  // connection 
  using (MySqlConnection connection = new MySqlConnection(_connectionstring))
  {
   //Open connection
   connection.Open();

   //Create command
   using (MySqlCommand command = new MySqlCommand("SELECT * FROM nested_category ORDER BY lft", connection))
   {
    //Read the table row
    using (MySqlDataReader reader = command.ExecuteReader())
    {
     //Read row values
     Node node;
     while (reader.Read())
     {
      node = new Node(); 
      node.CategoryId = Convert.ToDouble(reader["id"]);
      node.Text = reader["text"];
      node.Left = Convert.ToInt(reader["lft"]);
      node.Right = Convert.ToInt(reader["rgt"]);

      nodes.Add(node);
     }
    }
   }
  }

  if (nodes.Count == 0)
   return null;

  //Build the node hierarchy
  int nodeIndex = 0;
  root = BuildNodes(nodes, ref nodeIndex);
 }
 catch (Exception ex)
 {
  throw new Exception(ex);
 }

 return root;
}

Recursive Method


The last step is the recursive method to step through the list and convert it to a tree structure.  The trick here is to track 3 values, the row index and then the left and right values of the node in that row index.  The flow is hierarchy
  1. If a leaf node (a node without children) then return the node.
  2. If an internal node (a node with children) then
    1. Continue to step through the list and add children while the child's right value is less than the node's right value.

private Node BuildNodes(List<Node> nodes, ref int nodeIndex)
{
 
 Node node = nodes[nodeIndex];
 if ((node.Right - node.Left) == 1)
 {
  //Leaf
  return node;
 }
 else
 {
  //Internal node     
  Node child;
  int childIndex = node.Left++;
  while (childIndex + 1 < node.Right)
  {
   nodeIndex++;
   child = BuildNodes(nodes, ref nodeIndex);
   if (child != null)
   {
    childIndex = child.Right++;
    node.Nodes.Add(child);
   }
  }    
 }

 return node;
}

This will build up the hierarchy converted from a list.  It assumes that the left and right values are continuous and there are no numbers missing.  If your list filters the tree and there are gaps in the left and right values then good luck and let me know what you find.

Wednesday, December 9, 2015

Inserting and Deleting In a Nested Set Model

Storing tree data in a database can be tricky. There are multiple approaches you can take depending on your project's needs.  For one of my projects I added and removed nodes so infrequently (like once or twice a year) but required thousands of requests per day.  The nested set model was the perfect fit and I got an excited for the chance to play with a new technique that was such a shift in thinking to adjacency lists.  I'm not going to go into what the nested set model is, I'm assuming that since you are here you know something about it.  If not then I recommend the de facto blog here: http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/

I find using procedures for these tasks to be much easier than requiring users to wrap their head around the nested set model or to try and translate the code to the language in your client.

 

Inserting

One item missing from the blog is the actual code, sure it gives you snippets but not the entire picture.  For example inserting nodes is different if you are dealing with a leaf or internal node.  Just a small oversight that I thought I would resolve by creating a procedure.  It takes as input the id of the parent node that we want to insert and we pass the values for the new node.  The procedure tests if the parent is a leaf node and branches accordingly.

DELIMITER;;
CREATE PROCEDURE `node_insert`(parent_category_id INT, name VARCHAR(255))
BEGIN
 DECLARE parent_left, parent_right INT DEFAULT 0;
 SELECT lft INTO parent_left FROM nested_category 
  WHERE category_id = parent_category_id;
 SELECT rgt INTO parent_right FROM nested_category 
  WHERE category_id = parent_category_id;

 IF parent_right = (parent_left + 1) THEN
  SELECT @my_left := lft FROM nested_category 
   WHERE id = parent_category_id;
  UPDATE nested_category SET rgt = rgt + 2 
   WHERE rgt > @my_left;
  UPDATE nested_category SET lft = lft + 2  
   WHERE lft > @my_left;
  INSERT INTO nested_category(`name`, `lft`, `rgt`) 
   VALUES(name, @my_left  + 1, @my_left + 2);
 ELSE
  SELECT @my_right := rgt FROM nested_category 
   WHERE id = parent_category_id;
  UPDATE nested_category SET rgt = rgt + 2 
   WHERE rgt >= @my_right;
  UPDATE nested_category SET lft = lft + 2  
   WHERE lft >= @my_right;
  INSERT INTO nested_category(`name`, `lft`, `rgt`) 
   VALUES(name, @my_right, @my_right + 1);
 END IF;
END ;;
DELIMITER ;

To call the procedure we will need the id of the parent node and values.  For example if we were to add a node for smartwatches to the PORTABLE_ELECTRONICS that has an id of 6 we can call the procedure like this:

CALL node_insert(6, 'Smartwatch');

 

Deleting

This one is pretty easy. We take as input the id of the node we want to delete.

CREATE PROCEDURE `node_delete`(category_id INT)
BEGIN
 SELECT @my_left := `lft`, @my_right := `rgt`, @my_width := `rgt` - `lft` + 1
  FROM `nested_category`
  WHERE `category_id` = category_id;

 DELETE FROM `nested_category` 
  WHERE `lft` BETWEEN @my_left AND @my_right;

 UPDATE `nested_category` SET `rgt` = `rgt` - @my_width  
  WHERE `rgt` > @my_right;
 UPDATE `nested_category` SET `lft` = `lft` - @my_width 
  WHERE `lft` > @my_right;
END

To call the procedure we just need the id of the node.  For example if we decided that CD players were now obsolete we can delete the node "CD PLAYERS" which has an id of 9.


CALL node_delete(9);

Monday, October 13, 2014

A Real World Example of GWT 2.x, MySQL, and MVP: Part 4

Part 1: Getting Started
Part 2: Putting the Framework Together
Part 3: Finally Making the Page Content
Part 4: User Authentication

User Authentication

Now that we have our website up and running let's protect certain pages and restrict functionality that we wouldn't want to give to all users who visit your site with user authentication by using a Role Based Access Control (RBAC) approach. Don't know what RBAC is? You're best bet is to Google some tutorials and purchase some books because it can get real complicated real fast. Because there are so many approaches I will take the most basic and general approach which you can modify for whatever variant you wish to use.  Now at this time let me say it is VERY IMPORTANT that access to your site should be done over SSL, that means your website should be "https" and not "http". Information like passwords will be sent over in plain text and using SSL will fix a huge security layer on top of your website.

There are 3 basic principles of Role Based Access Control that we will need to know first:
  • User - An person (or service account) who can perform functions on your webpage.
  • Role - A job function that defines authority level.
  • Permission - The authority to someone to perform an operation.
The relationship between these principles looks like this:
Image source: http://www.isaca.org/Journal/Past-Issues/2008/Volume-3/Pages/JOnline-Role-Engineering-The-Cornerstone-of-RBAC1.aspx
Basically:
  • A User has one or many Roles
  • A Role has one or many Permissions
Next we need to decide what roles we will need and what type of access we would like to give to each of our roles. Having a good solid strategy ahead of time will make your security stronger and save you time later on.  In a real website we could add roles for super users, power users, super power users, each with their own levels of access, but for this example I will keep things very simple and have only 2 types of roles:
  • Administrator
  • User
Finally we should decide what each of these roles can and can't do.  For example we don't want an average user able to change things we don't want them to.  Remember a user can have multiple roles, so there is no need to duplicate some permissions. 
  • Administrator
    • View admin page
    • Add search engine
    • Edit search engine
    • Remove search engine
    • Edit user settings
  • User
    • Log-in
    • Edit own user settings
The basic idea is to be able to check to see if a user has permission to perform an action, not what role they have. We will do this in 2 ways. On the database side we will create a system that will use triggers and functions to check if a user has authority to perform an action, and on the client side we will store the list of permissions with the user object and perform checks on that list for a permission.

Create database tables

I love it when a plan comes together! Now let's get moving. To set this up in a database we will need 6 tables, 5 of them will store everything we will need for RBAC authentication, and 1 table to store user sessions. A session is how we will actually be authenticating a user after he or she logs into the system. A session should be unique per user and very difficult to predict and have an end date requiring the user to re-login at some point.  You can create each table manually or also download a SQL script here.
    Table 1: users
    Stores user information and any metadata you want to keep. Things like, address, city, state, gender, etc... Heck be as creative as you want because if you're lucky you can eventually sell this information to advertisers for a profit! Evil laugh! MWA-HA-HA-HA! (Seriously though don't be a jerk and do that.)
    CREATE TABLE `users` (
      `id` int(11) NOT NULL AUTO_INCREMENT,
      `username` varchar(99) NOT NULL COMMENT 'SYMC username',
      `password` varchar(99) NOT NULL COMMENT 'User password',
      `first_name` varchar(99) DEFAULT NULL COMMENT 'User first name',
      `last_name` varchar(99) DEFAULT NULL COMMENT 'User last name',
      `email` varchar(99) DEFAULT NULL COMMENT 'User email address',
      `created_by` int(11) NOT NULL COMMENT 'Created by',
      `created_on` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Date created on',
      `updated_by` int(11) DEFAULT NULL COMMENT 'Updated by',
      `updated_on` timestamp NULL DEFAULT NULL COMMENT 'Date updated on',
      `active` tinyint(1) DEFAULT '1' COMMENT 'Is the user active',
      PRIMARY KEY (`id`),
      UNIQUE KEY `id_UNIQUE` (`id`)
    ) ENGINE=InnoDB COMMENT='User of the system';
    
    

    Table 2: roles
    Stores the roles that can be assigned to a user.
    CREATE TABLE `roles` (
      `id` int(11) NOT NULL AUTO_INCREMENT,
      `name` varchar(45) DEFAULT NULL,
      `description` varchar(45) DEFAULT NULL,
      PRIMARY KEY (`id`)
    ) ENGINE=InnoDB COMMENT='User roles';
    

    Table 3: permissions
    Permissions hold the whole security together.  They tell us that user X can do Y with object Z. The name will be a short name to description what the permission allows and will be our alternate key, action will be something like "add", "remove", "modify", and presenter will be the webpage (object) we want to access.
    CREATE TABLE `permissions` (
      `id` int(11) NOT NULL AUTO_INCREMENT,
      `name` varchar(99) DEFAULT NULL,
      `action` varchar(45) DEFAULT NULL,
      `presenter` varchar(45) DEFAULT NULL,
      PRIMARY KEY (`id`)
    ) ENGINE=InnoDB COMMENT='User permissions';
    
    

    Table4: user_roles
    This will contain many-to-many references for users and their assigned roles. We'll also add some constraints to delete rows if a user or a role is ever deleted from their respective tables. That should keep this table nice and tidy.
    CREATE TABLE `user_roles` (
      `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'Unique identifier',
      `user_id` int(11) DEFAULT NULL COMMENT 'User identifier',
      `role_id` int(11) DEFAULT NULL COMMENT 'Role identifier',
      PRIMARY KEY (`id`),
      UNIQUE KEY `id_UNIQUE` (`id`),
      KEY `FK_user_roles_users_idx` (`user_id`),
      KEY `FK_user_roles_roles_idx` (`role_id`),
      CONSTRAINT `FK_user_roles_roles` FOREIGN KEY (`role_id`) REFERENCES `roles` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION,
      CONSTRAINT `FK_user_roles_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION
    ) ENGINE=InnoDB COMMENT='User roles';
    
    

    Table 5: role_permissions
    Ah, ah, ah, ah, table 5! Table 5! Not a Simpsons fan, eh? Never-mind then. This will contain many-to-many references for roles and their permissions. Again we'll add constraints to keep the table neat.
    CREATE TABLE `role_permissions` (
      `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'Unique identifier',
      `role_id` int(11) DEFAULT '0' COMMENT 'Role identifier',
      `permission_id` int(11) DEFAULT '0' COMMENT 'Permission identifier',
      PRIMARY KEY (`id`),
      UNIQUE KEY `id_UNIQUE` (`id`),
      KEY `FK_role_permissions_roles_idx` (`role_id`),
      KEY `FK_role_permissions_permissions_idx` (`permission_id`),
      CONSTRAINT `FK_role_permissions_permissions` FOREIGN KEY (`permission_id`) REFERENCES `permissions` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION,
      CONSTRAINT `FK_role_permissions_roles` FOREIGN KEY (`role_id`) REFERENCES `roles` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION
    ) ENGINE=InnoDB COMMENT='Role permissions';
    

    Table 6: user_sessions
    This table will contain user sessions. It will help us in our system security to track when a user has logged in, if the session is still active, and also provide some mechanism for metrics at a later date if we so wish.
    CREATE TABLE `user_sessions` (
      `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'Unique identifier',
      `user_id` int(11) NOT NULL DEFAULT '0' COMMENT 'User identifier',
      `session_id` varchar(45) DEFAULT NULL COMMENT 'Unique client session identifier',
      `ip_address` varchar(45) NOT NULL COMMENT 'User IP address',
      `created_on` timestamp NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Date created on',
      `ended_on` timestamp NULL DEFAULT NULL COMMENT 'Date session will end on',
      PRIMARY KEY (`id`)
    ) ENGINE=InnoDB;
    
    

    Create Some Triggers

    If you are not familiar with databases this might start getting a bit complicated, but don't worry it really isn't. We are going to create a trigger on the "users" table to automatically assign any new user the role "user" which will give them the ability to log into the system, edit their own user settings, and any other permissions we give the user role. This will save us the time from having to add this role manually every time a new user signs up or is registered in the system.
    DELIMITER ;;
    CREATE TRIGGER `users_AINS` AFTER INSERT ON `users` FOR EACH ROW
    BEGIN
        -- Get the user role id
        DECLARE role_id INT;
        SET role_id = (SELECT `id` FROM `roles` WHERE name = 'user'); 
    
        -- Insert the new user with a user role
        INSERT INTO `user_roles` (`user_id`, `role_id`) VALUES (NEW.id, role_id);
    END ;;
    DELIMITER ;
    

    Insert Test Data

    For the sake of getting something out of this tutorial what I will do is create a test user and set up our roles and permissions directly within the database. What should really happen is we create some administrator pages in GWT to manage users, roles, and permissions... and we will get to that (I think.)  In the mean-time let's set up some test data:

    Insert 1: roles
    INSERT INTO `roles` VALUES 
    (1,'administrator','Administrator of the system'),
    (2,'user','User of the system');
    

    Insert 2: permissions
    INSERT INTO `permissions` VALUES 
    (1,'allow_login','login','login'),
    (2,'allow_show_admin','show','admin'),
    (3,'allow_modify_engine','modify','engine'),
    (4,'allow_add_engine','add','engine'),
    (5,'allow_remove_engine','remove','engine'),
    (6,'allow_modify_user','modify','user'),
    (7,'allow_modify_self','modify','self');
    

    Insert 3: users
    INSERT INTO `users` VALUES 
    (1,'admin','test','Admin','Test','admin@mom.com',1,NULL,NULL,NULL,1),
    (2,'user','test','User','Test','user@mom.com',1,NULL,NULL,NULL,1);
    

    STOP RIGHT THERE! I wanna know right now. Before we go any further do you now have rows in the "user_roles" table? If so it should look like this:
    iduser_idrole_id
    112
    222
    This means that when we created the users above in "Insert 3" they were automatically assigned the "user" role by the trigger 'users_AINS'. Now we don't have to worry about making sure every user can log in when they register at our site. Cool!

    Insert 4: user_roles
    We already have some user roles thanks to the trigger, but now we just have to give the 'admin' user the 'administrator' role.
    INSERT INTO `user_roles` (`user_id`, `role_id`) VALUES 
    (1,1);
    

    Insert 5: role_permissions
    Do the following insert and it will assign the permissions to the roles that we decided above. Again normally we would have pages in an Administrator section on our website to manage what permissions belong to what roles, and I highly recommend building those, but once again let's just get some test data so we can play with this tutorial.
    • Administrator (role_id: 1)
      • View admin page (permission_id: 2)
      • Add search engine (permission_id: 4)
      • Edit search engine (permission_id: 3)
      • Remove search engine (permission_id: 5)
      • Edit user settings (permission_id: 6)
    • User (role_id: 2)
      • Log-in (permission_id: 1)
      • Edit own user settings (permission_id: 7)
    INSERT INTO `role_permissions` VALUES 
    (1,1,2),
    (2,1,3),
    (3,1,4),
    (4,1,5),
    (4,1,6),
    (5,2,1),
    (6,2,7);
    

    I think that gets us to a place where we have a normal user and an administrator. Sweet. Now let's start adding to our website.

    Refactoring the Server

    I'm glad you stepped through my tutorial on MySQL and GWT (if you haven't then right now might be a good time right now since that is where all the code I will be referencing is) but now that we are working on a much larger and scalable application we need to make some tiny adjustments to my previous code. Here is the dilemma, the authentication stuff and the database stuff are going to be used together a lot. What I want to avoid is having multiple RPC guys on my client, because that's not where I want to do all of the business logic for authentication. I want to check to see if a user has permission to insert a record or get certain records before sending that data back to the client. So what I want to avoid is this:


    Instead I want to add another layer of abstraction by creating a class to group all off the back-end services on the server-side in order to deal with all of the business logic, such as the authentication crap we're working on now and the data layer. This will have the benefit of separation of concerns and saving time with RPC calls.

     To start we need a few things. I will first create a custom Exception called "NotAuthenticatedException" which will help communicate specific failures from my server to my client. Of course you can skip this and send everything as a plain-old Exception, but what if you want to branch your logic depending on exactly what the problem was on the server-side? Maybe I want to limit the number of times a user has to authenticate? It will be easier to add this up by catching NotAuthenticatedExceptions than looking at something like the error message. So here we go:

    NotAuthenticatedException.java
    package com.example.searchengines.shared;
    
    public class NotAuthenticatedException extends Exception {
    
        private static final long serialVersionUID = 1L;
    
        public NotAuthenticatedException() {
            this("Login failed; Invalid username or password");
        }
    
        public NotAuthenticatedException(String arg0) {
            super(arg0);
        }
    
        public NotAuthenticatedException(Throwable arg0) {
            super(arg0);
        }
    
        public NotAuthenticatedException(String arg0, Throwable arg1) {
            super(arg0, arg1);
        }
    }
    

    Now let's add modify and add new methods to our DBConnection and DBConnectionAsync services that we created in my previous blog to check user authentication and log a user out of the system:

    DBConnection.java
    package com.rappa.lapizza.client.service;
    
    public interface DBConnection extends RemoteService
    {
        /**
         * Authenticates the User with the specified username and password
         * @param username User's name
         * @param password User's password
         * @return User
         * @throws Exception
         */
        public User authenticateUser(String username, String password) throws NotAuthenticatedException, Exception;
    
        /**
         * Logs out a user session
         * @param jSessionId User session to log out
         * @throws Exception 
         */
        public void logout(String jSessionId) throws Exception;
    

    DBConnectionAsync.java
    package com.example.searchengines.client.service;
    
    public interface DBConnectionAsync
    {
        /**
         * Authenticates the User with the specified username and password
         * @param username User's name
         * @param password User's password
         * @param callback Async return of the User
         * @return User
         * @throws Exception
         */
        public void authenticateUser(String username, String password, AsyncCallback<User> callback) throws NotAuthenticatedException, Exception;
    
        /**
         * Logs out a user session
         * @param jSessionId User session to log out
         * @param callback Nothing
         */
        public void logout(String jSessionId, AsyncCallback<Void> callback) throws Exception;
    

    Sorry MySQLConnection but we are stripping you of your rank of RemoteServiceServlet. Hand over your badge and your gun. Let's remove the "extends RemoteServiceServlet" but keep "implements DBConnection" in order to no longer use this class as our remote service.

    MySQLConnection.java
    package com.example.searchengines.server;
    
    public class MySQLConnection extends RemoteServiceServlet implements DBConnection {
    

    Now let's add a new class to act as a repository for the multiple back-end server processes. If I thought ahead with the previous tutorials I would maybe have named this better, but whatever fuck-it, I'll change it all later. This class will basically wrap the MySQLConnection class and pass most of the results on.

    DBConnectionImpl.java
    package com.example.searchengines.server;
    
    /**
     * @author Austin_Rappa
     *
     */
    public class DBConnectionImpl extends RemoteServiceServlet implements DBConnection {
    
        private MySQLConnection mysql = new MySQLConnection();
        
        /**
         * Authenticates the User with the specified username and password
         * @param username User's name
         * @param password User's password
         * @return User
         * @throws Exception
         */
        public User authenticateUser(String username, String password) throws Exception, NotAuthenticatedException {
            User user = null;
            
            user = mysql.authenticateUser(username, password);
            if (user != null) {
                //Store session id
                HttpServletRequest httpServletRequest = this.getThreadLocalRequest();
                HttpSession session = httpServletRequest.getSession(true);
                String jSessionId = session.getId();
                session.setAttribute("JSESSIONID", jSessionId);
                
                //Insert database session
                mysql.insertUserSession(user, jSessionId, getThreadLocalRequest().getRemoteAddr());
            }
            
            return user;
        }
    
        /**
         * Logs out a user session
         * @param jSessionId User session to log out
         * @throws Exception 
         */
        public void logout(String jSessionId) throws Exception {
            //Remove session id from http header
            HttpServletRequest httpServletRequest = this.getThreadLocalRequest();
            HttpSession session = httpServletRequest.getSession();
            session.removeAttribute("JSESSIONID");
            
            //Logout from database session
            mysql.logout(jSessionId);
        }
    
        /**
         * Gets the SearchEngines
         * @return SearchEngine
         * @throws Exception
         */
        public List<SearchEngine> getSearchEngines() throws Exception {
            return mysql.getSearchEngines();
        }
    }
    

    Finally we have to update the web.xml file to use the new DBConnectionImpl.java class instead of MySQLConnection.java class.

    web.xml
        <servlet>
            <servlet-name>mySQLConnection</servlet-name>
            <servlet-class>com.example.searchengines.server.DBConnectionImpl</servlet-class>
        </servlet>
    

    Permissions in the Website

    First thing I am going to do is create an enum with my list of permissions. You don't have to do this and in-fact can be a pain as your website grows and strings work just fine as permission names shouldn't change. For me I don't mind. Add an enum to your 'shared' package called 'Permission.java'.

    Permissions.java
    package com.example.searchengines.shared;
    
    public enum Permission {
        ALLOW_LOGIN,
        ALLOW_SHOW_ADMIN,
        ALLOW_MODIFY_ENGINE,
        ALLOW_ADD_ENGINE,
        ALLOW_REMOVE_ENGINE,
        ALLOW_MODIFY_USER,
        ALLOW_MODIFY_SELF
    }
    

    Next edit your user class in the "shared" package to ad a list of permissions. If you chose to not use an enum for your permissions just change the List type to String and be on your way. What we will do is store a list of the permissions assigned to the user. This will allow our presenter classes to check to see if this user has permission to perform an action or show a control.

    User.java
    package com.example.searchengines.shared;
    
    public class User implements IsSerializable {
    
        private List<Permission> permissions;
    
        ...
    
        /**
         * Gets the user permissions
         * @return the permissions
         */
        public List<Permission> getPermissions() {
            return permissions;
        }
    
        /**
         * Sets the users permissions
         * @param permissions the permissions to set
         */
        public void setPermissions(List<Permission> permissions) {
            this.permissions = permissions;
        }
    

    Let's standardize on the length of time we would like the session to be active. 24 hours seems good so let's go with that. If you want to change to a week just change that "1" at the end of the formula to a "7".

    Property.java
    package com.example.searchengines.client;
    
    public class Property {
        
        public static final long SessionDuration = (1000 * 60 * 60 * 24) * 1; //duration remembering login.     
    


    The last step for the server-side code is adding methods to the MySQLConnection.java class to connect to the database, look up the username and password (which is right now stored as plain-text but we will change later) and return a User object with the user's information, otherwise raise an NotAuthenticatedException if we could not authenticate.

    MySQLConnection.java
    package com.example.searchengines.server;
    
    public class MySQLConnection implements DBConnection {
    
        /**
         * Authenticates the User with the specified username and password
         * @param username User's name
         * @param password User's password
         * @return User
         * @throws Exception
         */
        public User authenticateUser(String username, String password) throws Exception, NotAuthenticatedException {        
            Connection conn = null;
            PreparedStatement pstmt = null;
            ResultSet result = null;
            User user = null;
    
            try {
                conn = getConnection();
                
                boolean authenticated = true;
                //TODO: Authenticate encrypted password.
                
                if (authenticated) {
                    pstmt = conn.prepareStatement("SELECT * FROM users WHERE username = ? AND password = ?");
                    pstmt.setString(1, username);
                    pstmt.setString(2, password);
                    result = pstmt.executeQuery();
                    
                    //Check for null results
                    if (result.isBeforeFirst() ) {
                        //Loop through results
                        while (result.next()) {
                            //Create return object
                            user = new User();
                            user.setId(result.getDouble("id"));
                            user.setUserName(result.getString("username"));
                            user.setFirstName(result.getString("first_name"));
                            user.setLastName(result.getString("last_name"));
                            user.setEmail(result.getString("email"));
                            user.setPermissions(this.getUserPermissions(result.getDouble("id")));
                        }
                    }
                    else {
                        throw new NotAuthenticatedException();
                    }
                }
                else {
                    throw new NotAuthenticatedException();
                }
            } catch (SQLException sqle) {
                logger.error("SQL error in authenticateUser(" + username + "): " + sqle.getMessage() + "\n" + sqle.getStackTrace().toString());
                throw new Exception(sqle);
            } finally {
                result.close();
                pstmt.close();
                conn.close();
            }
    
            return user;
        }
        
        /**
         * Logs a user login session
         * @param user User who logged into session
         * @param session_id Session identifier
         * @param ip_address User's IP address
         * @throws Exception     
         */
        void insertUserSession(User user, String session_id, String ip_address) throws Exception {
            Connection conn = null;
            PreparedStatement pstmt = null;
    
            try {
                // Get database connection
                conn = getConnection();            
    
                // Insert new user
                pstmt = conn.prepareStatement("INSERT INTO user_sessions (user_id, session_id, ip_address, created_on, ended_on) VALUES (?, ?, ?, ?, ?)", Statement.RETURN_GENERATED_KEYS);
                pstmt.setDouble(1, user.getId());
                pstmt.setString(2, session_id);
                pstmt.setString(3, ip_address);
                pstmt.setTimestamp(4, new Timestamp(new java.util.Date().getTime()));
                pstmt.setTimestamp(5, new Timestamp(new java.util.Date().getTime() + Property.SessionDuration));
                 
                //Execute sql statement
                if (pstmt.executeUpdate() == 0) {            
                    logger.error("Error in insertUserSession(): Bad log of session.");
                }
            } catch (SQLException sqle) {
                logger.error("Error in insertUserSession(): " + sqle.getMessage() + "\n" + sqle.getStackTrace().toString());
                throw new Exception(sqle);
            } catch (Exception e) {
                throw new Exception(e);
            } finally {
                // Cleanup
                pstmt.close();
                conn.close();
            }
        }
        
        /**
         * Gets the permission list of a user
         * @param user_id User identifier
         * @throws Exception     
         */
        private List<Permission> getUserPermissions(Double user_id) throws Exception {
            Connection conn = null;
            PreparedStatement pstmt = null;
            ResultSet result = null;
            List<Permission> permissions = new ArrayList<Permission>();
    
            try {
                // Get database connection
                conn = getConnection();            
    
                //Get user permissions
                pstmt = conn.prepareStatement("SELECT permissions.name FROM user_roles LEFT JOIN role_permissions ON role_permissions.role_id = user_roles.role_id LEFT JOIN permissions ON permissions.id = role_permissions.permission_id WHERE user_roles.user_id = ?");
                pstmt.setDouble(1, user_id);
                 
                //Execute sql statement
                result = pstmt.executeQuery();
                while (result.next()) {
                    Permission permission = Permission.valueOf(result.getString("name").toUpperCase());
                    permissions.add(permission);
                }
            } catch (SQLException sqle) {
                logger.error("Error in getUserPermissions(): " + sqle.getMessage() + "\n" + sqle.getStackTrace().toString());
                throw new Exception(sqle);
            } catch (Exception e) {
                throw new Exception(e);
            } finally {
                // Cleanup
                result.close();
                pstmt.close();
                conn.close();
            }
            
            return permissions;
        }
        
         /**
          * Logs out a user session
          * @param jSessionId User session to log out
          * @throws Exception 
          */
        public void logout(String jSessionId) throws Exception {
            Connection conn = null;
            PreparedStatement pstmt = null;
    
            try {            
                // Get database connection
                conn = getConnection();            
    
                // Insert new user
                pstmt = conn.prepareStatement("UPDATE user_sessions SET ended_on = NOW() WHERE session_id = ?");
                pstmt.setString(1, jSessionId);
                 
                //Execute sql statement
                if (pstmt.executeUpdate() == 0) {
                    throw new Exception("Bad log of session.");
                }
            } catch (SQLException sqle) {
                logger.error("Error in logout(): " + sqle.getMessage() + "\n" + sqle.getStackTrace().toString());
                throw new Exception(sqle);
            } catch (Exception e) {            
                logger.error("Error in logout(): " + e.getMessage() + "\n" + e.getStackTrace().toString());
                throw new Exception(e);
            } finally {
                // Cleanup
                pstmt.close();
                conn.close();
            }   
        }
    


    Modifying the User Interface

    Let's create some events! Events in GWT are used to broadcast a notification to the rest of the code in your project and any code that is listening for that notification will be called. What we want to do is to notify our header page (and any other page that is listening) that the user has successfully logged-in and take actions to show a different collection of buttons, like a log out button.  In your "client" package created a sub-package called "event" and create the following classes:

    LogInEvent.java
    package com.example.searchengines.client.event;
    
    import com.google.gwt.event.shared.GwtEvent;
    
    public class LogInEvent extends GwtEvent<LogInEventHandler>{
        
          public static Type<LogInEventHandler> TYPE = new Type<LogInEventHandler>();
          
          public LogInEvent() {
          }
    
          @Override
          public Type<LogInEventHandler> getAssociatedType() {
            return TYPE;
          }
    
          @Override
          protected void dispatch(LogInEventHandler handler) {
            handler.onLogIn(this);
          }
    }
    

    LogInEventHandler.java
    package com.example.searchengines.client.event;
    
    import com.google.gwt.event.shared.EventHandler;
    
    public interface LogInEventHandler extends EventHandler {
          void onLogIn(LogInEvent event);
    }
    

    LogOutEvent.java
    package com.example.searchengines.client.event;
    
    import com.google.gwt.event.shared.GwtEvent;
    
    public class LogOutEvent extends GwtEvent<LogOutEventHandler>{
        
          public static Type<LogOutEventHandler> TYPE = new Type<LogOutEventHandler>();
          
          public LogOutEvent() {
          }
    
          @Override
          public Type<LogOutEventHandler> getAssociatedType() {
            return TYPE;
          }
    
          @Override
          protected void dispatch(LogOutEventHandler handler) {
            handler.onLogOut(this);
          }
    }
    

    LogOutEventHandler.java
    package com.example.searchengines.client.event;
    
    import com.google.gwt.event.shared.EventHandler;
    
    public interface LogOutEventHandler extends EventHandler {
          void onLogOut(LogOutEvent event);
    }
    
    

    This is where we add the most important piece, the guy that ties this all together on the client-side, the "REMEMBER_ME" cookie. Create a presenter for the log-in page in your "presenter" package. This will hold the log-in page logic. For our view we'll need a text box for the username and password, a button to accept the form, and a button to take the user to a password recovery page. If the user authentication comes back from the server okay then we will create a cookie called "REMEMBER_ME" that has an expiration date, this way if a user returns to your site we can check if that session is still active and retrieve the session.

    LogInPresenter.java
    package com.example.searchengines.client.presenter;
    
    public class LoginPresenter implements Presenter {
    
        public interface Display {
            HasClickHandlers getLogin();
            HasClickHandlers getForgotPassword();
            HasKeyPressHandlers getKeyPressUsername();
            HasKeyPressHandlers getKeyPressPassword();
            HasText getUsername();
            HasText getPassword();
            Widget asWidget();
            void showError(String errormessage);
            void hideError();
        }
    
        private final DataFactory factory;
        private final Display display;
    
        /**
         * 
         * @param factory
         * @param view
         */
        public LoginPresenter(DataFactory factory, Display view) {
            this.factory = factory;
            this.display = view;
        }
        
        /**
         * 
         */
        public void bind() {
    
            display.getLogin().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    try {
                        if (!display.getUsername().getText().isEmpty() && !display.getPassword().getText().isEmpty()) {
                            factory.getRpcService().authenticateUser(display.getUsername().getText(), display.getPassword().getText(), new AsyncCallback<User>() {
                                public void onFailure(Throwable caught) {
                                    display.showError("Error authenticating user. " + caught.getMessage());
                                }
        
                                public void onSuccess(final User user) {
                                    if (user != null) {
                                        //Set session cookie for 1 day expire. Set secure to false while in development but true in production.
                                        String sessionID = Cookies.getCookie("JSESSIONID");
                                        Date expires = new Date(System.currentTimeMillis() + Property.SessionDuration);
                                        Cookies.setCookie("REMEMBER_ME", sessionID, expires, null, "/", false);
                                        
                                        //Set user in the data factory
                                        factory.setUser(user);
    
                                        //Broadcast log-in notification. Presenters like the HeaderPresenter are listening for this.                            
                                        factory.getEventBus().fireEvent(new LogInEvent());              
                                        
                                        //Go to home page
                                        History.newItem("home");
                                    }
                                    else {
                                        display.showError("Username or password is invalid.");
                                    }
                                        
                                }                        
                            });
                        }
                        else {
                            display.showError("Please enter your username and password.");
                        }
                    } 
                    catch (Exception caught) {
                        display.showError("Error authenticating user. " + caught.getMessage());
                    }
                }            
            });
            
            display.getKeyPressUsername().addKeyPressHandler(new KeyPressHandler() {
                public void onKeyPress(KeyPressEvent event) {
                    if (event.getNativeEvent().getKeyCode() == KeyCodes.KEY_ENTER) {
                        display.getLogin().fireEvent(new ClickEvent(){});
                    }
                }            
            });
            
            display.getKeyPressPassword().addKeyPressHandler(new KeyPressHandler() {
                public void onKeyPress(KeyPressEvent event) {
                    if (event.getNativeEvent().getKeyCode() == KeyCodes.KEY_ENTER) {
                        display.getLogin().fireEvent(new ClickEvent(){});
                    }
                }            
            });        
        }
        
        /**
         * 
         */
        public void go(final HasWidgets container) {
            bind();
            container.clear();
            container.add(display.asWidget());
        }
    }
    
    

    Now let's create the view that will contain our display widgets.

    LoginView.java
    package com.example.searchengines.client.view;
    
    public class LoginView extends View implements LoginPresenter.Display {
        
        private Button forgotpasswordButton = new Button(constants.forgotpassword());
        private Button loginButton = new Button(constants.login());
        private TextBox usernameTextBox = new TextBox();
        private PasswordTextBox passwordTextBox = new PasswordTextBox();
        private Label errorLabel = new Label();
    
        public LoginView() {    
        
            //Main panel
            VerticalDivPanel mainPanel = new VerticalDivPanel();
            mainPanel.addStyleName("login");
            initWidget(mainPanel);    
            
            //Define controls
            Label usernameLabel = new Label(constants.loginUsername());
            usernameLabel.setStyleName("bodytext-bold");
            Label passwordLabel = new Label(constants.loginPassword());
            passwordLabel.setStyleName("bodytext-bold");
                    
            loginButton.setStyleName("loginbutton");
            loginButton.setTitle("Log in");
            
            forgotpasswordButton.setStyleName("hyperlink");
            
            //Form table
            FlexTable loginTable = new FlexTable();
            loginTable.setStyleName("loginform");
            
            //Add controls to form
            loginTable.setWidget(0, 0, usernameLabel);
            loginTable.setWidget(0, 1, usernameTextBox);
            loginTable.setWidget(1, 0, passwordLabel);                
            loginTable.setWidget(1, 1, passwordTextBox);
            loginTable.setWidget(2, 1, loginButton);
            loginTable.setWidget(3, 1, forgotpasswordButton);
            mainPanel.add(loginTable); 
            
            Scheduler.get().scheduleDeferred(new Command() {
                public void execute() {
                    usernameTextBox.setFocus(true);
                }
              });
            
            //Error message
            this.hideError();
            errorLabel.setStyleName("errortext");
            mainPanel.add(errorLabel);        
        }
        
        public Widget asWidget() {
            return this;
        }        
    
        public HasClickHandlers getLogin() {
            return this.loginButton;
        }
    
        public HasText getUsername() {
            return this.usernameTextBox;
        }
    
        public HasText getPassword() {
            return this.passwordTextBox;
        }
    
        public void showError(String errormessage) {
            errorLabel.setText(errormessage);
            errorLabel.setVisible(true);
        }
    
        public void hideError() {
            errorLabel.setText("");
            errorLabel.setVisible(false);
        }
    
        public HasKeyPressHandlers getKeyPressUsername() {
            return this.usernameTextBox;
        }    
        
        public HasKeyPressHandlers getKeyPressPassword() {
            return this.passwordTextBox;
        }
    
        public HasClickHandlers getForgotPassword() {
            return this.forgotpasswordButton;
        }
    }
    

    Now that we have our Log-in header and presenter let's modify our project so our users can browse to those pages when they click the LogIn button on the header. Open the HeaderPresenter class and change "TODO 10" with the highlighted code below that inserts a new browser history and highlights the LogIn header button.

    HeaderPresenter.java
    package com.example.searchengines.client.presenter;
    
    public class HeaderPresenter implements Presenter {
    
        ...
        
        /**
         * Bind events and actions
         */
        public void bind() {
            
            ...
            
            display.getLogIn().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    History.newItem("login");
                    display.setSelected();
                }
            });
    

    For the last part of the log-in page we need to tell the AppController class to call the LoginPresenter when the Log-in header button is clicked and your site's url changes with the "login" token added to the history.

    AppController.java
    package com.example.searchengines.client;
    
    public class AppController implements Presenter, ValueChangeHandler<String> {
    
        ...
    
        public void onValueChange(ValueChangeEvent<String> event) {
            ...        
                
                // For every page you create set the presenter here
                if (token.getPage().equals("home"))    {
                    presenter = new HomePresenter(factory, new HomeView());
                }        
                ...
                }
                else if (token.getPage().equals("login")) {
                    presenter = new LoginPresenter(factory, new LoginView());
                }            
    

    If you debug the page, now you should be able to see the log-in page when you click the button on the header. Also notice the "login" token in your url. That is a result of adding the "History.newItem()" in the HeaderPresenter and the AppController class picking up that change in the history, branching on that history token value, and displaying the login page.

    Admin only pages

    Let's say we want to give users who have elevated privileges access to areas of your website where changes to the site's data can be made. Modify HeaderPresenter.java and add functionality for a new button for a future administrator page and to listen for log-in and log-out events. If you followed the previous tutorials (which you should have) then this will replace the TODO #4 for the log-out button and TODO #5 for listening for log-in events. We'll also add a check if the data factory's user object is not null.

    HeaderPresenter.java
    public class HeaderPresenter implements Presenter {
    
        public interface Display {
            ...
            HasClickHandlers getAdmin();
            ...
        }
    
        public void bind() {
            ...
            display.getAdmin().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    History.newItem("admin");
                    display.setSelected();
                }
            });
    
            display.getLogOut().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) { 
                    doLogout();
                }
            });
            
            factory.getEventBus().addHandler(LogInEvent.TYPE, new LogInEventHandler() {
                public void onLogIn(LogInEvent event) {
                    doLogIn(factory.getUser());
                } 
            });
        }
    
        public void go(final HasWidgets container) {
            bind();
            container.clear();
            container.add(display.asWidget());
            if (factory.getUser() != null) {
                this.doLogIn(factory.getUser());
            }
            display.setSelected();
        }
    
        private void doLogIn(User user) {
            display.setData(factory.getUser());
        }
    
        private void doLogout() {
            try {
                String sessionId = Cookies.getCookie("REMEMBER_ME");
                factory.getRpcService().logout(sessionId, new AsyncCallback() {
                    public void onFailure(Throwable caught) {
                }
    
                public void onSuccess(Void result) {
                    Cookies.removeCookie("REMEMBER_ME");
                    factory.setUser(null);
                    display.setData();
                    factory.getEventBus().fireEvent(new LogOutEvent());
                    History.fireCurrentHistoryState();  
                }});
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    

     Nearing the home stretch. Let's just modify the HeaderViewto show our new "Admin" button and to display it only if the user has the "allow_show_admin" permission. Oh wow see how that all comes together now? You can have 50 roles that have the "allow_show_admin" permission and instead of listing them all we just need to know if that user can get to the admin page and that is all. Sweet!

    HeaderView.java
    package com.example.searchengines.client.view;
    
    public class HeaderView extends View implements HeaderPresenter.Display {
                
        private final Button adminButton = new Button(constants.admin());
        
        public void setData(User user) {
            //Clear right panel
            rightPanel.clear();
            
            //Hello user
            Label userLabel = new Label(constants.welcome() + " " + user.getFirstName());
            userLabel.setStyleName("headerpaneltext");
            rightPanel.add(userLabel);
            
            Label seperatorLabel = new Label("|");
            seperatorLabel.setStyleName("headerpaneltext");
            rightPanel.add(seperatorLabel);
            
            if (user.getPermissions().contains(Permission.ALLOW_SHOW_ADMIN)) {
                adminButton.setStyleName("headerbutton");
                rightPanel.add(adminButton);
                
                Label seperatorLabelAdmin = new Label("|");
                seperatorLabelAdmin.setStyleName("headerpaneltext");
                rightPanel.add(seperatorLabelAdmin);
            }
                
            logoutButton.setStyleName("headerbutton");
            rightPanel.add(logoutButton);
        }
    
    
        public HasClickHandlers getAdmin() {
            return this.adminButton;
        }
    }
    

    Almost forgot. Add a new constant to SearchEnginesConstants.java for the display text of our administrator button.

    SearchEnginesConstants.java
    public interface SearchEnginesConstants extends Constants {
            
        ...
        @DefaultStringValue("Admin")
        String admin();
        ...
    

    Let's take a quick look what our website can do so far with authentication:
    • Display log-in page
    • Authenticate a user
    • Retrieve information about that user
    • Assign permissions to a user based on their role
    • Display a new admin button on the header when user has permission
    • When admin header button is clicked, the "admin" token is added to the history.
    Now let's take the last step and display the admin page, only if the user has permission to do so. In the AppControler, the class that controls what page to display in our webpage, we will add some logic to show an Administrator page, only if the user who has logged in, has the "ALLOW_SHOW_ADMIN" permission, otherwise send them to the home page.

    package com.example.searchengines.client;
    
    public class AppController implements Presenter, ValueChangeHandler<String> {
    
        ...
    
        public void onValueChange(ValueChangeEvent<String> event) {
            ...        
                
                // For every page you create set the presenter here
                if (token.getPage().equals("home"))    {
                    presenter = new HomePresenter(factory, new HomeView());
                }        
                ...
                }
                else if (token.getPage().equals("admin")) {
                    if (factory.getUser() != null && factory.getUser().getPermissions().contains(Permission.ALLOW_SHOW_ADMIN)) {
                        //TODO 9: Create administrator page
                    }
                    else {
                        presenter = new HomePresenter(factory, new HomeView());
                    }
                }               
    

    Now when you debug the page as a Web Application try authenticating with your test user and you should not see an "Admin" button on the header but have a "Log Out" button available. However if you authenticate with your test admin user then not only should you see the "Admin" button on the header but you can click that button and your site's url will change with an "admin" token now on the history like so:

    http://127.0.0.1:8888/SearchEngines.html?gwt.codesvr=127.0.0.1:9997#admin

    If you see this then that will indicate that your test admin user does indeed have permission to view the administrator page, we just haven't built it yet.  So in the next tutorial let's build the administrator page, a user registration page, add some more roles and permissions, and learn how to encrypt user passwords in the database.

    References


    http://csrc.nist.gov/rbac/rbac-std-ncits.pdf
    http://code.google.com/p/google-web-toolkit-incubator/wiki/LoginSecurityFAQ
    https://www.owasp.org/index.php/Authentication_Cheat_Sheet
    https://www.owasp.org/index.php/Session_Management_Cheat_Sheet
    https://code.google.com/p/yfs-health/source/browse/trunk/yfs/src/com/varun/yfs/server/login/



    Friday, October 10, 2014

    SAXON: Fatal Error! Cannot find a matching 1-argument function named {http://exslt.org/common}node-set(). There is no Saxon extension function with the local name node-set

    I was working on a DITA-OT plugin when I encountered this guy:

    Fatal Error! Cannot find a matching 1-argument function named {http://exslt.org/common}node-set(). There is no Saxon extension function with the local name node-set

    I am using the latest version of SAXON HE which apparently does not support EXSLT.
    http://www.saxonica.com/feature-matrix.html
    However you can pay $50 bucks or more for the PE and EE versions. Or you can save yourself some green and download SAXON 9.1.0.8 which was the last version to support EXSLT. You can download the JAR file here and put it on your classpath. Hint: The download link is next to the floppy drive icon.
    Download saxon-9.1.0.8.jar

    For a DITA project you'll also need the following jar:
    Download saxon-dom-9.1.0.8.jar

    And if you are using Eclipse you can set the classpath by doing the following:

    1. On the file menu click: Window > Preferences > Ant > Runtime > Classpath tab > Global entries
    2. Remove the existing versions of SAXON from the classpath
    3. Click the "Add External Jars" button
    4. Add "saxon-9.1.0.8.jar" and "saxon-dom-9.1.0.8.jar"



    Thursday, August 28, 2014

    Tracking Downloads To Your Eclipse Plug-in Update Site

     Source

    Eclipse Update Site
    artifacts.xsl

    Introduction

    My company wanted to add a plug-in to Eclipse for our internal bug-tracking system. No problem I say and I build that sucker. I used Spring Tool Suite 3.5.1 (which is just Eclipse 4.3 with a fancy paint job) with Maven, and deployed to a Linux Red Hat server, mostly following the vogella tutorials. But now they want to know adoption rates and that will be part of my annual review! Holy shit I better find out how to get some metrics quick! Oh look here is the p2 documentation: Equinox p2 download stats

    Umm... okay well thanks for nothing really. The idea is to add 2 properties to artifacts.xml, which is packaged inside of artifacts.jar, that tells the update site to perform an HTTP HEAD request (it's like a HTTP GET request but without body content) whenever your plug-in is downloaded with the url you provide. However if you use Eclipse (for your Eclipse plug-in) then we are immediately going to run into some problems. The p2 instructions would like us to modify artifacts.xml, however that file is generated automajically and there are no way to set properties in your project and have it appear in artifacts.xml. Well not yet at least: Eclipse Project 4.5 M1 - New and Noteworthy. Sonofabitch!
    So what is suggested is to perform the following steps:
    1. Unpack artifacts.jar
    2. Modify artifacts.xml and add the 'p2.statsURI' and 'download.stats' properties.
    3. Repackage artifacts.jar

     Configuring the Eclipse Update Site

    Aw fuck, so I have to waste time and write scripts now? Don't bang your head on the table I've already figured it out. That's what I'm here for.  First we have to create an XSLT transform to deal with adding the properties to artifacts.xml that point to your url. The transform will have 2 parameters, which can be passed on the command-line, that will set the value of the url that gets called when the plug-in is downloaded. That url is ultimately a combination of the 'p2.statsURI' and 'download.stats' properties.  So if set
    • p2.statsURI=www.foo.com/updatesite/stats/
    •  download.stats=index.jsp
    The website that will get called is 'www.foo.com/updatesite/stats/index.jsp'

    So in your update site project create a file called 'artifacts.xsl'. For my example I created a subfolder called 'xslt' and placed artifacts.xsl there.

    artifacts.xsl
    <?xml version="1.0"?>
    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    
      <xsl:output method="xml" version="1.0" encoding="UTF-8"  indent="yes" />
    
      <xsl:param name="p2.statsURI"></xsl:param>
      <xsl:param name="download.stats"></xsl:param>
    
      <xsl:template match="@*|node()">
        <xsl:copy>
          <xsl:apply-templates select="@*|node()" />
        </xsl:copy>
      </xsl:template>
    
      <xsl:template match="/repository/properties">
        <xsl:element name="properties">
          <xsl:attribute name="size">
            <xsl:value-of select="@size + 1" />
          </xsl:attribute>
          <xsl:apply-templates select="node()" />
          <xsl:element name="property">
            <xsl:attribute name="name">p2.statsURI</xsl:attribute>
            <xsl:attribute name="value">
              <xsl:value-of select="$p2.statsURI" />
            </xsl:attribute>
          </xsl:element>
        </xsl:element>
      </xsl:template>
    
      <xsl:template match="artifact[@classifier='org.eclipse.update.feature']">
        <xsl:element name="artifact">
          <xsl:attribute name="classifier">
            <xsl:value-of select="@classifier"></xsl:value-of>
          </xsl:attribute>
          <xsl:attribute name="id">
            <xsl:value-of select="@id"></xsl:value-of>
          </xsl:attribute>
          <xsl:attribute name="version">
            <xsl:value-of select="@version"></xsl:value-of>
          </xsl:attribute>
          <xsl:apply-templates select="node()" mode="artifact" />
        </xsl:element>
      </xsl:template>
      
      <xsl:template match="properties" mode="artifact">
        <xsl:element name="properties">
          <xsl:attribute name="size">
            <xsl:value-of select="@size + 1" />
          </xsl:attribute>
          <xsl:apply-templates select="node()" mode="artifact" />
          <xsl:element name="property">
            <xsl:attribute name="name">download.stats</xsl:attribute>
            <xsl:attribute name="value">
              <xsl:value-of select="$download.stats" />
            </xsl:attribute>
          </xsl:element>
        </xsl:element>
      </xsl:template>
      
      <xsl:template match="property" mode="artifact">
        <xsl:copy>
          <xsl:apply-templates select="@*|node()" />
        </xsl:copy>
      </xsl:template>
    
    </xsl:stylesheet>
    


    Cool. Next we need to modify the update site's pom.xml to add the Maven Ant plugin. This will allow us to hook into a maven life-cycle phase and run some Ant tasks that we specify. OBEY ME ANT! MAVEN IS NO LONGER YOUR MASTER! Right. So in a nutshell we will demand Ant to do the following after the update site has been compiled and packaged into their jars.
    1. Unpack artifacts.jar in a temporary folder called 'output'. Artifacts.jar contains only 1 file, artifacts.xml.
    2. Modify artifacts.xml and add the 'p2.statsURI' and 'download.stats' properties using the transform above, artifacts.xsl. In the param nodes set your site's values in the expression attributes.
      1. For 'p2.statsURI' we will be eventually creating a subfolder in your update site's location called 'stats' that will contain the files used to track downloads. Don't forget the slash at the end.
      2. For 'download.stats' this will be the page that will process the HTTP HEAD request. For my example I will use JSP but you can use PHP, Perl, or whatever I don't care I'm not your boss.
    3. Repackage artifacts.jar.
    4. Remove the temp folder 'output'.

    pom.xml
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-antrun-plugin</artifactId>
      <version>1.7</version>
      <executions>
        <execution>
          <id>prepare</id>
          <phase>package</phase>
          <configuration>
            <tasks>
              <echo message="Add download stats to artifacts.jar" />
              <unzip src="${project.build.directory}/repository/artifacts.jar" dest="output/" />
              <xslt in="output/artifacts.xml" out="output/artifacts2.xml" style="${basedir}/xslt/artifacts.xsl">
                <param name="p2.statsURI" expression="http://yoursite/updatesite/stats/"/>
                <param name="download.stats" expression="insert.jsp"/>
              </xslt>
              <move file="output/artifacts2.xml" tofile="output/artifacts.xml" overwrite="true" />
              <zip destfile="${project.build.directory}/repository/artifacts.jar"  basedir="output" update="true" />
              <delete dir="output"/>
            </tasks>
          </configuration>
          <goals>
            <goal>run</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
    



    Processing the HTTP Request

    Yay we're done! Not so fast. We're not done. Not at all. Now we have to process that page the update site will call when your plug-in is downloaded. My example will be in JSP with a MySQL database to store the data. Also here is an example in PHP: p2-stats-recorder

    First let's set up the MySQL database to hold the information of the folks who download the plug-in.  Create a new database called 'p2_stats'.
    CREATE DATABASE p2_stats;
    

    Create a user account named 'p2' with the password 'p2' (or you can be smart and name it something stronger, please be smart!)
    CREATE USER 'p2'@'localhost' IDENTIFIED BY 'p2';
    

    Next let's create our lone table.
    CREATE TABLE `installations` (
    `id` INT(11) NOT NULL AUTO_INCREMENT,
    `package` VARCHAR(255),
    `version` VARCHAR(255),
    `os` VARCHAR(255),
    `host` VARCHAR(255),
    `created_on` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY(`id`));
    

    Finally let's give the 'p2' user access to the 'installations' table.
    GRANT INSERT, SELECT, UPDATE, DELETE ON `installations` TO 'p2'@'localhost';
    

    That's it. This is identical to schema.sql in the p2-stats-recorder, except I use 'created_on' as the name of my timestamp column, removed the ui column that I personally won't need, and added a primary key column... because... you know... database normalization.

    Now on your server in your update site directory create a folder called 'stats'.  This is where all of our scripts to track downloads will be.

    Next is to create the script that will process the HTTP HEAD request. As I mentioned before I am using JSP which requires the MySQL Java connector to be installed. Just go here Download Connector/J, unzip that guy and place the jar in a your updatesite folder in the location 'WEB-INF\lib', with 'WEB-INF' being a sibling of 'features', 'plugins', and your newly created 'stats' folder. MySQL says to fuck around with the path but fuck that shit as this method is preferred for so many reasons. Also don't be stupid like me and waste hours trying to figure out why a connection to MySQL cannot be established and first make sure that your MySQL server is started.

    Create a new file in your 'stats' folder called 'insert.jsp' or whatever value you gave your 'download.stats' xslt parameter. Essentially what we want to do is establish a connection to our 'p2_stats' MySQL database, then insert a new record with data gathered from the HTTP header. Here is the code for that guy.:

    insert.jsp
    <%@ page import="java.sql.*" %> 
    <%@ page import="java.io.*" %> 
    
    <html>
    <head>
    <title>My Eclipse Plug-in Download Counter</title>
    <body>
    <h2>Insert download record</h2>
    
    <%
    
    try {
      String connectionURL = "jdbc:mysql://localhost:3306/p2_stats";
      Connection connection = null; 
      PreparedStatement pstatement = null;
    
      Class.forName("com.mysql.jdbc.Driver").newInstance(); 
      connection = DriverManager.getConnection(connectionURL, "p2", "p2");
      if(!connection.isClosed()) {    
        String queryString = "INSERT INTO installations(package, version, os, host) VALUES(?, ?, ?, ?)";                   
        pstatement = connection.prepareStatement(queryString);
        pstatement.setString(1, request.getParameter("project.name"));
        pstatement.setString(2, request.getParameter("project.version"));
        pstatement.setString(3, request.getParameter("os.name"));
        pstatement.setString(4, request.getRemoteHost());
                    
        int updateQuery = pstatement.executeUpdate();
        if (updateQuery != 0) {
          out.println("Successfully inserted record.<BR/>");
          out.println("project.name: " + request.getParameter("project.name") + "<BR/>");
          out.println("project.version: " + request.getParameter("project.version") + "<BR/>");
          out.println("os.name: " + request.getParameter("os.name") + "<BR/>");
          out.println("host: " + request.getRemoteHost() + "<BR/>");
        }
      }
      connection.close();
    }catch(Exception ex){
        out.println("Unable to connect to database. " + ex);
    }
    
    %> 
    
    </body>
    </html>
    
    

    Now we can test the page by opening a browser and using the url:
    http://yoursite/updatesite/stats/insert.jsp

    You can ensure a record was created by looking at your 'installations' table in the database. You can also check your server logs. For example I am using Tomcat so in the directory '/var/log/tomcat' I can look in the file 'localhost_access_log' and I should see an entry:

    "HEAD /eclipse-updatesite/stats/insert.jsp HTTP/1.1" 200 240

    Hooray it worked! You now have a new record in your database. Right now some of the columns are blank and I'm sorry I may have deceived you... but look the basics are working and I promise I will fix that in a later "advanced reporting" section.

    That Later Advanced Reporting Section

    AW FUCK THIS SECTION IS WRONG. COME BACK TOMORROW AND I'LL HAVE IT FIXED.

    Let's say you want to track some information about who and what was downloaded. Seems reasonable. All that has to be done is the following:
    1. Add url parameters to the download.stats page in the pom.xml 
    2. Capture those parameters in our download.stats page
    3. Insert the parameters into the database. 
    Luckily the code to capture the parameters and insert them into the database were already included above but I saved this part for a later section because we want to be smart about it and URL encode the values we pass and that requires further details. First thing we need to do is add project properties to the update site's pom.xml so we can modify them with URL encoded values. For example, let's say we want to track the project name, the project version, and the user's operating system. Then we can add some properties that will hold those values which we can then manipulate later.

    pom.xml
      <properties>
        <project.name.url>${project.name}</project.name.url>
        <project.version.url>${project.version}</project.version.url>
      </properties>
    



    Cool. Next let's add a new plugin to run groovy scripts called gmaven-plugin which will allow us to run Java commands. So we will take our properties we added above and be able to URL encode them or perform any other string manipulation you would need. You might notice I am adding the os.name.url variable here and didn't include it above. That is because the os.name is a System property and any the other guys are Maven properties which will be null at this point and that is why we added them above.

    pom.xml
          <plugin>
            <groupId>org.codehaus.gmaven</groupId>
            <artifactId>gmaven-plugin</artifactId>
            <version>1.5</version>
            <executions>
              <execution>
                <id>setup-groovy</id>
                <phase>initialize</phase>
                <goals>
                  <goal>execute</goal>
                </goals>
                <configuration>
                  <source>
                    project.properties["project.name.url"] = java.net.URLEncoder.encode(project.properties['project.name.url'])
                    project.properties["project.version.url"] = java.net.URLEncoder.encode(project.properties['project.version.url'])
                    project.properties["os.name.url"] = java.net.URLEncoder.encode(System.properties['os.name'])                
                  </source>              
                </configuration>
              </execution>
            </executions>
          </plugin>
    


    Finally let's append URL parameters to the download.stats xslt parameter.  So find that guy again in your pom.xml and make the change from this:
    <param name="download.stats" expression="insert.jsp" />
    
    to this:
    <param name="download.stats" expression="insert.jsp?project.name=${project.name.url}&amp;project.version=${project.version.url}&amp;os.name=${os.name.url}" />
    

    And that's it! You can add as many parameters as you like, just remember to separate them with &amp; and not just an & since we are working with xml.  Then whatever parameter you add, alter your p2_stats table to add a new column that you want to map to it, and change the sql statement in your download.stats to insert your new parameter to your new column.

    I just want to say this took me two days to work through.  If this saved you any time I suggest you stop what you are doing and enjoy whatever time I saved you with friends, family, a good book, binge watch that Netflix show, whatever.


    Sunday, December 29, 2013

    A Real World Example of GWT 2.x, MySQL, and MVP: Part 3

    Part 1: Getting Started
    Part 2: Putting the Framework Together
    Part 3: Finally Making the Page Content
    Part 4: User Authentication 

    Finally Making the Page Content


    This is a continuation of my previous blog.

    This tutorial is a how-to guide to help you build an actual, scalable, website with GWT and a MySQL back-end.  Right now if you try to Run or Debug your project it isn't going to do a lot as there is much to still set up.  Let's begin by starting from the bottom and continue to work our way up and by the end of this article we should have a site that we can launch from Eclipse with a home page, an about page, and getting data from our MySQL database.  W00t!

    This page will follow the MVP architecture and is not intended to teach it to you.  In short you will have 2 classes for a single page, a presenter that handles all of the business logic, and a view that handles the layout.  If you want to know more than please first read an MVP tutorial like this.

    I will not be using UiBinder feature since, GWT 2.5.1 the current version as of this article, is total crap when working with the MVP architecture.  Seriously it is if MVP and internationalization were an afterthought when Google added it to GWT.  I will create a parallel tutorial page for those using the UiBinder feature.

    We will first review the header then work our way on to the different content pages.

    Header Presenter

    The header presenter is business logic for the header component of our page.  It contains an inner interface called "Display" that assists in communication with the header view.  The constructor has a parameter of the data factory and the header view object.

    package com.example.searchengines.client.presenter;
    
    import com.example.searchengines.client.DataFactory;
    import com.example.searchengines.shared.User;
    import com.google.gwt.event.dom.client.ClickEvent;
    import com.google.gwt.event.dom.client.ClickHandler;
    import com.google.gwt.event.dom.client.HasClickHandlers;
    import com.google.gwt.user.client.History;
    import com.google.gwt.user.client.ui.HasWidgets;
    import com.google.gwt.user.client.ui.Widget;
    
    /**
     * A presenter for the header
     */
    public class HeaderPresenter implements Presenter {
    
        public interface Display {
            HasClickHandlers getHome();
            HasClickHandlers getAbout();
            HasClickHandlers getLogIn();
            HasClickHandlers getLogOut();
            HasClickHandlers getRegister();
            void setData();
            void setData(User user);
            void setSelected();
            Widget asWidget();
        }
        
        private final DataFactory factory;
        private final Display display;
        
        /**
         * A presenter for the header
         * @param factory The data factory
         * @param view The header view
         */
        public HeaderPresenter(DataFactory factory, Display view) {
            this.factory = factory;
            this.display = view;
        }
        
        
        /**
         * Bind events and actions
         */
        public void bind() {
            
            display.getHome().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    History.newItem("home");
                    display.setSelected();
                }            
            });
            
            display.getAbout().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    History.newItem("about");
                    display.setSelected();
                }
            });    
            
            display.getLogIn().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    //TODO 10: Add login to history
                }
            });
            
            display.getLogOut().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {    
                    //TODO 5: Fire LogOutEvent           
                }
            });
            
            display.getRegister().addClickHandler(new ClickHandler() {
                public void onClick(ClickEvent event) {
                    History.newItem("register");
                    display.setSelected();
                }
            });
            
            //TODO 6: Listen for LogInEvents
        }
    
        
        /**
         * Go forth and build the page
         * @param contents Html content node
         */
        public void go(final HasWidgets contents) {
            bind();
            contents.clear();
            contents.add(display.asWidget());
            display.setSelected();
        }
    }
    

    Header View

    The header view implements the Display interface of the header presenter, which helps to allow communication between the header and the presenter.  It has 2 setData() methods, one for when the site is accessed anonymously or when a user has logged in.

    package com.example.searchengines.client.view;
    
    import com.example.searchengines.client.presenter.HeaderPresenter;
    import com.example.searchengines.client.widget.HorizontalDivPanel;
    import com.example.searchengines.client.widget.VerticalDivPanel;
    import com.example.searchengines.shared.User;
    import com.google.gwt.event.dom.client.HasClickHandlers;
    import com.google.gwt.user.client.History;
    import com.google.gwt.user.client.ui.Button;
    import com.google.gwt.user.client.ui.Label;
    
    /**
     * Header page
     */
    public class HeaderView extends View implements HeaderPresenter.Display {
                
        private final Button homeButton = new Button(constants.home());
        private final Button aboutButton = new Button(constants.about());
        private final Button loginButton = new Button(constants.login());
        private final Button logoutButton = new Button(constants.logout());
        private final Button registerButton = new Button(constants.register());
        private final HorizontalDivPanel rightPanel = new HorizontalDivPanel();
        
    
        /**
         * Header page
         */
        public HeaderView() {
            VerticalDivPanel mainPanel = new VerticalDivPanel();
            mainPanel.addStyleName("mainpanel");
            initWidget(mainPanel);            
            
            //Title
            Label titleLabel = new Label(constants.name());
            titleLabel.setStyleName("titletext");
            mainPanel.add(titleLabel);
            
            //Control Panel
            HorizontalDivPanel controlPanel = new HorizontalDivPanel();
            controlPanel.addStyleName("headerpanel");
            mainPanel.add(controlPanel);
                
                homeButton.setStyleName("headerbutton");
                controlPanel.add(homeButton);
    
                aboutButton.setStyleName("headerbutton");
                controlPanel.add(aboutButton);
                
                //Right panel
                controlPanel.add(rightPanel);
                controlPanel.setChildStyleName("alignright", controlPanel.getWidgetIndex(rightPanel));
                
            //Show default buttons
            setData();
        }
        
        
        /**
         * Set data for users not logged in
         */
        public void setData() {
            //Clear right panel
            rightPanel.clear();    
            
            //Login button
            registerButton.setStyleName("headerbutton");
            rightPanel.add(registerButton);
            
            Label seperatorLabel = new Label("|");
            seperatorLabel.setStyleName("headerpaneltext");
            rightPanel.add(seperatorLabel);
            
            //Login button
            loginButton.setStyleName("headerbutton");
            rightPanel.add(loginButton);
        }
            
        
        /**
         * Sets data for a user who is logged in
         */
        public void setData(User user) {
            //Clear right panel
            rightPanel.clear();
            
            //Hello user
            Label userLabel = new Label(constants.welcome() + " " + user.getFirstName());
            userLabel.setStyleName("headerpaneltext");
            rightPanel.add(userLabel);
            
            Label seperatorLabel = new Label("|");
            seperatorLabel.setStyleName("headerpaneltext");
            rightPanel.add(seperatorLabel);
                
            logoutButton.setStyleName("headerbutton");
            rightPanel.add(logoutButton);
        }
        
        
        /**
         * Sets a CSS style on the selected button
         */
        public void setSelected() {
            //Clear selected
            homeButton.removeStyleDependentName("selected");
            aboutButton.removeStyleDependentName("selected");
            loginButton.removeStyleDependentName("selected");
            logoutButton.removeStyleDependentName("selected");
            registerButton.removeStyleDependentName("selected");
            
            //Set selected
            String selected = History.getToken();        
            if (selected.equals("home")) {
                homeButton.setStyleDependentName("selected", true);
            }    
            else if (selected.equals("about")) {
                aboutButton.setStyleDependentName("selected", true);
            }
            else if (selected.equals("login")) {
                loginButton.setStyleDependentName("selected", true);
            }
            else if (selected.equals("logout")) {
                logoutButton.setStyleDependentName("selected", true);
            }
            else if (selected.equals("register")) {
                registerButton.setStyleDependentName("selected", true);
            }
        }
        
    
        /**
         * Home button
         */
        public HasClickHandlers getHome() {
            return this.homeButton;
        }
    
    
        /**
         * About button
         */
        public HasClickHandlers getAbout() {
            return this.aboutButton;
        }
    
    
        /**
         * Log in button
         */
        public HasClickHandlers getLogIn() {
            return this.loginButton;
        }
    
    
        /**
         * Log out button
         */
        public HasClickHandlers getLogOut() {
            return this.logoutButton;
        }
    
    
        /**
         * Registration button
         */
        public HasClickHandlers getRegister() {
            return registerButton;
        }
    }
    


    Adding the Header to the App Controller

    Now that we have our header all defined and ready for prime time we just have to attach it to the HTML header node in the App Controller.  So open AppController.java and find TODO comment number 1:
    //TODO 1: Add Header presenter and view
    
    and replace it with an instantiation of the header presenter, passing in the data factory and a new header view:
    Presenter presenter = new HeaderPresenter(factory, new HeaderView());
    presenter.go(header);
    

    So your final setHeader() method should look like this:
    public void setHeader(final HasWidgets header) {
        try {    
            this.header = header;
            Presenter presenter = new HeaderPresenter(factory, new HeaderView());
            presenter.go(header);
        }
        catch (Exception ex) {
            ex.printStackTrace();
        }
    }
    

    Home Presenter

    package com.example.searchengines.client.presenter;
    
    import java.util.List;
    import com.example.searchengines.client.DataFactory;
    import com.example.searchengines.shared.SearchEngine;
    import com.google.gwt.event.dom.client.HasClickHandlers;
    import com.google.gwt.user.client.ui.HasWidgets;
    import com.google.gwt.user.client.ui.Widget;
    import com.google.gwt.view.client.SingleSelectionModel;
    
    /**
     * A presenter for the home page
     */
    public class HomePresenter implements Presenter {
    
        public interface Display {
            HasClickHandlers getAdd();        
            SingleSelectionModel<SearchEngine> getSelectionModel();
            void setData(List<SearchEngine> rowData);
            void showError(String message);
            Widget asWidget();
        }
    
        private final DataFactory factory;
        private final Display display;
    
        
        /**
         * A presenter for the home page
         * @param factory
         * @param view
         */
        public HomePresenter(DataFactory factory, Display view) {
            this.factory = factory;
            this.display = view;
        }
    
        
        /**
         * Bind events and actions
         */
        public void bind() {
        }
         
        
        /**
         * Go forth and build the page
         */
        public void go(final HasWidgets container) {
            bind();
            container.clear();
            container.add(display.asWidget());
            this.fetchData();
        }
        
        
        /**
         * Fetch page information
         */
        private void fetchData() {
            try {                    
                //TODO 3: Let's use MySQL!
            } catch (Exception e) {
                e.getStackTrace();
            }
        }
    }
    

    Home View

    The home view will use my custom StarRatingStarRatingCell and StarRatingColumn widgets from my previous blogs where you can add them to your client's "widget" package.

    package com.example.searchengines.client.view;
    
    import java.util.List;
    import com.example.searchengines.client.presenter.HomePresenter;
    import com.example.searchengines.client.widget.HorizontalDivPanel;
    import com.example.searchengines.client.widget.StarRating;
    import com.example.searchengines.client.widget.StarRatingColumn;
    import com.example.searchengines.client.widget.VerticalDivPanel;
    import com.example.searchengines.shared.SearchEngine;
    import com.google.gwt.cell.client.ClickableTextCell;
    import com.google.gwt.cell.client.FieldUpdater;
    import com.google.gwt.event.dom.client.HasClickHandlers;
    import com.google.gwt.user.cellview.client.CellTable;
    import com.google.gwt.user.cellview.client.Column;
    import com.google.gwt.user.cellview.client.TextColumn;
    import com.google.gwt.user.client.Window;
    import com.google.gwt.user.client.ui.Button;
    import com.google.gwt.view.client.DefaultSelectionEventManager;
    import com.google.gwt.view.client.ProvidesKey;
    import com.google.gwt.view.client.SingleSelectionModel;
    
    /**
     * Home page
     */
    public class HomeView extends View implements HomePresenter.Display {    
        
        private Label errorLabel = new Label();
        private Button addButton = new Button(constants.add());
        private CellTable<SearchEngine> cellTable;
        private SingleSelectionModel<SearchEngine> selectionModel;
        
        /**
         * Home page
         */
        public HomeView() {
            
            //Main panel
            VerticalDivPanel mainPanel = new VerticalDivPanel();
            mainPanel.addStyleName("mainpanel");
            mainPanel.addStyleName("homepage");
            initWidget(mainPanel);
    
            errorLabel.setStyleName("errorlabel");
            errorLabel.setVisible(false);
            mainPanel.add(errorLabel);
            
            //TODO 7: Add new search engine button
            
            //Welcome
            VerticalDivPanel dataPanel = new VerticalDivPanel();
            dataPanel.addStyleName("datapanel");
            mainPanel.add(dataPanel);
    
                //Table key provider for sorting
                ProvidesKey<SearchEngine> keyProvider = new ProvidesKey<SearchEngine>() {
                    public Object getKey(SearchEngine item) {                  
                        return (item == null) ? null : item.getId();       
                    }     
                };                                                 
                
                //Data table
                cellTable = new CellTable<SearchEngine>(keyProvider);
                cellTable.setWidth("100%");    
                mainPanel.add(cellTable);
                
                //Add a selection model to handle user selection.     
                selectionModel = new SingleSelectionModel<SearchEngine>(keyProvider);     
                cellTable.setSelectionModel(selectionModel, DefaultSelectionEventManager.<SearchEngine>createDefaultManager());             
            
                    //Create name column.
                    Column<SearchEngine, String> nameColumn = new Column<SearchEngine, String>(new ClickableTextCell()) {
                        @Override
                        public String getValue(SearchEngine searchEngine) {
                            return searchEngine.getName();
                        }     
                    };
                    nameColumn.setFieldUpdater(new FieldUpdater<SearchEngine, String>() {   
                        public void update(int index, SearchEngine searchEngine, String value) {     
                            Window.open(searchEngine.getUrl(),"_blank","");
                        } 
                    }); 
                    nameColumn.setCellStyleNames("hyperlink");
                    cellTable.addColumn(nameColumn, constants.headername());    
                    
                    //Create url column.
                    TextColumn<SearchEngine> urlColumn = new TextColumn<SearchEngine>() {
                        @Override
                        public String getValue(SearchEngine searchEngine) {
                            return searchEngine.getUrl();
                        }     
                    }; 
                    cellTable.addColumn(urlColumn, constants.headerurl());
                    
                    //Category column.
                    TextColumn<SearchEngine> categoryColumn = new TextColumn<SearchEngine>() {
                        @Override
                        public String getValue(SearchEngine searchEngine) {
                            return searchEngine.getCategory().toString();
                        }     
                    }; 
                    cellTable.addColumn(categoryColumn, constants.headercategory());
                            
                    //Rating column
                    StarRatingColumn<SearchEngine> ratingColumn = new StarRatingColumn<SearchEngine>() {
                        @Override    
                        public StarRating getValue(SearchEngine searchEngine) {
                            return new StarRating(searchEngine.getRating(), 10, true);                    
                        }                                  
                    };
                    cellTable.addColumn(ratingColumn, constants.headerrating());    
                    
                    //TODO 8: Admin controls
        }
        
        
        /**
         * Add button
         */
        public HasClickHandlers getAdd() {
            return this.addButton;
        }
    
        
        /**
         * The table selection model
         */
        public SingleSelectionModel<SearchEngine> getSelectionModel() {
            return selectionModel;
        }
    
    
        /**
         * Set table data
         * @param data The data to set
         */
        public void setData(List<SearchEngine> data) {
            cellTable.setRowData(data);
        }
    
    
        /**
         * Show error message
         */
        public void showError(String message) {
            errorLabel.setText(message);
            errorLabel.setVisible(true);
        }
    }
    

    Adding the Home Page to the App Controller

    Now that we have our home component all defined we just have to attach it to the HTML contents node.  So open AppController.java and find TODO comment number 2:
    //TODO 2: For every page you create set the presenter here
    
    and replace it with an instantiation of the header presenter, passing in the data factory and a new header view:
    if (token.getPage().equals("home")) {
        presenter = new HomePresenter(factory, new HomeView());
    } 
    

    So your onValueChange() method should look like this:
    public void onValueChange(ValueChangeEvent<String> event) {
        Token token = new Token(event.getValue());
            
        if (token.getPage() != null) {
            Presenter presenter = null;            
                
            // For every page you create set the presenter here
            if (token.getPage().equals("home"))    {
                presenter = new HomePresenter(factory, new HomeView());
            }        
                
            //Load presenter
            if (presenter != null) {
                presenter.go(contents);            
            }
        }
    }
    


    About Presenter

    We will create an about page, which will just be a page with a little text blurb just so we have another page to click around to.
    package com.example.searchengines.client.presenter;
    
    import com.example.searchengines.client.DataFactory;
    import com.google.gwt.user.client.ui.HasWidgets;
    import com.google.gwt.user.client.ui.Widget;
    
    /**
     * A presenter for the about page
     */
    public class AboutPresenter implements Presenter {
    
        public interface Display {
            Widget asWidget();
        }
    
        @SuppressWarnings("unused")
        private final DataFactory factory;
        private final Display display;
        
        /**
         * A presenter for the about page
         * @param factory
         * @param view
         */
        public AboutPresenter(DataFactory factory, Display view) {
            this.factory = factory;
            this.display = view;
        }
        
        /**
         * Bind events and actions
         */
        public void bind() {
        }
          
        /**
         * Go forth and build the page
         */
        public void go(final HasWidgets container) {
            bind();
            container.clear();
            container.add(display.asWidget());
        }
    }
    

    About View

    Before we can create our about view we have to first add a new string constant to SearchEngineConstants.java in our "client" package.
    @DefaultStringValue("This is a website for my mom to help her find things on the web.")
    String abouttext();
    

    Now we can use it in our view.
    package com.example.searchengines.client.view;
    
    import com.example.searchengines.client.presenter.AboutPresenter;
    import com.example.searchengines.client.widget.VerticalDivPanel;
    import com.google.gwt.user.client.ui.Label;
    
    /**
     * About page
     */
    public class AboutView extends View implements AboutPresenter.Display {    
    
        /**
         * About page
         */
        public AboutView() {
            
            //Main panel
            VerticalDivPanel mainPanel = new VerticalDivPanel();
            mainPanel.addStyleName("mainpanel");
            mainPanel.addStyleName("aboutpage");
            initWidget(mainPanel);
            
            Label aboutLabel = new Label(constants.abouttext());
            mainPanel.add(aboutLabel);        
        }    
    }
    

    Adding the About Page to the App Controller

    In the HeaderPresenter.java class you will see we have already added a click handler for the about button to set the history to "about".  This will change the url in the browser from "www.mysite.com/home" to "www.mysite.com/about" and if you remember the App Controller is listening for changes to the history where we can then set what page is called depending on the current history value.  All we have to do is go back to AppController.java to the TODO comment number 2 and add an else if statement to set the presenter to an AboutPresenter.

    // For every page you create set the presenter here
    if (token.getPage().equals("home")) {
        presenter = new HomePresenter(factory, new HomeView());
    }        
    else if (token.getPage().equals("about")) {
        presenter = new AboutPresenter(factory, new AboutView());
    }
    

    Getting Data From MySQL

    Now we need to add a a way for us to retrieve a list of active search engines to fill our table on the home page.  Add the following method to your MySQLConnection.java class in the "server" package:

    /**
    * Gets a list of search engines
    * @return An ArrayList of SearchEngine
    * @throws Exception
    */
    public List<SearchEngine> getSearchEngines() throws Exception {
        List<SearchEngine> returnObjects = new ArrayList<SearchEngine>();
        Connection conn = null;
        PreparedStatement pstmt = null;
        ResultSet result = null;
    
        try {
            conn = getConnection();
            pstmt = conn.prepareStatement("SELECT * FROM search_engines WHERE is_active = TRUE ORDER BY rating DESC");
            result = pstmt.executeQuery();
            while (result.next()) {
                SearchEngine searchEngine = new SearchEngine();
                searchEngine.setId(result.getDouble("id"));
                searchEngine.setName(result.getString("name"));
                searchEngine.setUrl(result.getString("url"));
                searchEngine.setCategory(Category.fromInteger(result.getInt("category_id")));
                searchEngine.setRating(result.getInt("rating"));
                searchEngine.setActive(result.getBoolean("is_active"));
                returnObjects.add(searchEngine);
            }
        } catch (SQLException sqle) {
            logger.error("SQL error in getSearchEngines(): " + sqle.getMessage() + "\n" + sqle.getStackTrace().toString());
        } finally {
            result.close();
            pstmt.close();
            conn.close();
        }
    
        return returnObjects;
    }
    

    Add this method to your DBConnection.java class in the client's "services" package:

    /**
     * Gets a list of search engines
     * @return An ArrayList of BestRating
     * @throws Exception
     */
    public List<SearchEngine> getSearchEngines() throws Exception;
    

    And finally add this method to your DBConnectionAsync.java class in the client's "services" package:

    /**
     * Gets a list of search engines
     * @param callback Async return of the SearchEngine List
     * @return An ArrayList of SearchEngine
     * @throws Exception
     */
    public void getSearchEngines(AsyncCallback<List<SearchEngine>> callback) throws Exception;
    

    Now that we have a way to query data from our MySQL database and transfer it to our client let's go ahead an hook it up.  Open HomePresenter.java in your client's "presenter" package and find the TODO comment number 3:
    //TODO 3: Let's use MySQL!
    
    and replace it with a call to the method we just created through the data factory.
    try {                    
        factory.getRpcService().getSearchEngines(new AsyncCallback<List<SearchEngine>>() {
            public void onFailure(Throwable caught) {
                display.showError("Failure getting the search engines: " + caught.getMessage());
            }
    
            public void onSuccess(List<SearchEngine> result) {
                display.setData(result);
            }
        });
    } catch (Exception e) {
        display.showError("Error getting the search engines: " + e.getMessage());
    }
    

    So your fetchData() method should look like this:
    private void fetchData() {
            try {                    
                factory.getRpcService().getSearchEngines(new AsyncCallback<List<SearchEngine>>() {
                    public void onFailure(Throwable caught) {
                        display.showError("Failure getting the search engines: " + caught.getMessage());
                    }
    
                    public void onSuccess(List<SearchEngine> result) {
                        display.setData(result);
                    }
                });
            } catch (Exception e) {
                display.showError("Error getting the search engines: " + e.getMessage());
            }
        }
    


    Stylin'

    Look, I suck at CSS so don't judge me alright.  Place this in your SearchEngines.css file in your project's "war" folder:
    /** --- Html --- */
    html {height:100%;}
    body {height:100%;}
    div {display:block;}
    table {border-collapse:collapse; border-spacing:0;}
    
    /** --- DivPanels --- */
    .vertical {vertical-align: top;    margin: 0px; padding: 0px; width: 100%;}
    .vertical > ul {list-style: none; margin: 0px; padding: 0px; width: 100%;}
    .vertical > ul > li {display: block; text-decoration: none; margin: 0px; padding:0px 0px; overflow:hidden;}
    .horizontal {padding: 0px; margin: 0px;    width: 100%;}
    .horizontal > ul {float: left; list-style: none; padding: 0px; margin: 0px; width: 100%;}
    .horizontal > ul > li {float: left;    display: block;    text-decoration: none; margin: 0px;    padding: 0px;}
    
    /** --- Main Panels --- */
    .alignright {float:right!important;}
    .mainpanel {background:#fff9fe; min-width:780px; max-width:1260px; min-height:100%; margin:0px auto; padding:10px}
    .errortext {color:red;}
    
    /** -- Header -- */
    .headerpanel {height:30px; background:#eeeeee; border:2px solid #cccccc; width:1240px;}
    .headerbutton {border:none; margin:0px 2px; padding:6px 15px; color:#black; cursor:pointer; background:transparent;}
    .headerbutton:hover {background:#dddddd;}
    .headerbutton-selected {background:#fffaa5;}
    .headerbutton-selected:hover {background:#fffaa5;}
    .headerpaneltext {padding:6px 3px; color:black; background:transparent;}
    .titletext {font-size:200%; margin:25px 0px 5px 0px;}
    

    Debug Time!

    Holy crap! Quick, right click your project and choose Debug As > Web Application, then in the Development Mode tab right click http://127.0.0.1:8888/SearchEngines.html?gwt.codesvr=127.0.0.1:9997  and choose Open or Open With and then whatever Chrome browser Chrome you Chrome prefer.  Hopefully you will see a page that looks something like this:


    Hooray! I'm not an idiot! (At least when it comes to GWT.)  Don't worry if the page seems slow to load, that is just the debugger and will change once you place it on an actual web server.  I'm glad you took the time and stepped through each of my tutorials... or you could have cheated and downloaded the source here: Download Source

    Now let's add some authentication!