Saturday, December 11, 2010

A Simple Approach to Secure AJAX Image Loading

The goal of this post isn't to provide the details for creating a secure website from start to finish, as such a post would need to be far longer than I have any interest in writing. Instead, I hope this post gives you an idea of where to start. I use Unix for my testing, and Linux for my deployment, so I will mention how some of this might differ from a Windows machine, as the concepts here are more important than the actual code, which I will keep fairly generic. Also, I use PHP for all of my server code, but again, this should be fairly easily to translate to other server scripting languages such as ASP, JSP, Ruby, etc.

The Goal:
You need to create a site that loads images based on a user's account information that can't be accessed without proper credentials, such as an asset manager that pulls a user's logos and photos and displays them as part of a larger page.

The Problem:
First, it has to be secure. Second, you want it to be AJAX driven.

What Not To Do:
Don't rely on simple obfuscation methods to keep people out. Also, preventing what most servers refer to as "page indexing" does not satisfy our needs.

It is very important to know that storing images in randomly generated folder names does not provide adequate security, even if you then randomize the contents' filenames, nor does configuring your web server to prevent viewing folder contents in a browser. Both of these methods are great to implement, but should not be your only means of security.

A Basic Understanding Of Web Servers:
If you already understand how web servers work, skip this section.
I am assuming you are using Apache, or something similar. I'm not very well versed in other web servers, but hopefully the concepts here are broad enough that they can easily be ported. An important note: when I say web server, I am referring to the piece of software used to deliver content to users over HTTP or HTTPS.

Web servers use configuration files to determine what files are accessible via the web (which means either the HTTP or HTTPS protocol). On Apache, that is provided with the DocumentRoot directive, as such:

DocumentRoot /var/www/mywebpage


If this server was set up to deliver files for the domain name thegrassnull.com then the web server would map files in the folder /var/www/mywebpage to the domain thegrassynull.com, so typing thegrassynull.com/index.html into your browser tells the web server to find the file called 'index.html' on the server's hard driver in the folder /var/www/mywebpage (if you are unfamiliar with this style of folder paths, this is what is used in systems such as Unix and Linux. A Microsoft Windows equivalent would be something like C:\www\mywebpage)

Here's the important thing to understand: If we put files in the /var/www/mywebpage folder, the can be accessed by a browser provided a person knows (OR GUESSES) the file or folder names. If, however, we put a file in the /var/www folder (so, one level up from where our web page is kept), the browser has no way to reach it. Under normal circumstances, the web server has no idea what is in that folder, and doesn't care.

The Groundwork:
1) Create a folder to store the actual images
So, let's say we create a new folder /var/www/secureContents

The web server doesn't inherently know what is in that folder, and browsers (ie, the users that connect to your site) have no way of accessing it. To push this data to our users' browsers, we need to write a server script that will act as our data broker and retrieve these files. An important thing to keep in mind is that the web server runs as a user, and that user will need permission to access all of the files we plan on retrieving. The implementation details of this depend on the operating system your web server is running on, and I won't go into that (unless a demand for that exists).

2) Create the database that will contain the user information
I will leave the details of the user database up to you, but the key elements are the following:
user name
password
securepath
fileList -- Optional (we'll get to this in our PHP code later)


Password should NEVER BE HUMAN READABLE TEXT. EVER. Depending on where you live, you may have access to functions such as SHA, or MD5. Using those, you can create a hash of your passwords to store in the database. Again, that is outside the scope of this tutorial, but information can easily be found using your favorite search engine.


Securepath in this case could be something like /var/www/secureContents/user1 The important thing is that the folder exists prior to your attempt to access data in it, and that it is not a folder within your DocumentRoot.

3) Create the server script
Here is the framework for the server code that we will be creating. We will add more to this as we go along. Let's call this file Services.php and let's save it in the folder /var/www/mywebpage/php

//This code will provide data geared towards AJAX delivery.
//depending on how you implemented your user database, this code may be different
//start by creating a function for authenticating users

function Authenticate($username, $password) {
   $con = mysql_connect("localhost","databaseuser","databaseuserpassword");
   //these variables depend on YOUR configuration
   mysql_select_db("my_db", $con);
   //my_db is the scheme (in MySql) that you are using to store the user information table
   $result = mysql_query("SELECT secureFolder FROM Users where username = \"" . addslashes($username) . "\" AND password = \"" . md5($password) . "\"" );
   //if using mysql, look into mysql_real_escape_string instead of addslashes in the above code
   $numberOfResults = mysql_num_rows($result);
   //$numberOfResults MUST BE EXACTLY 1 TO BE A CORRECT USERNAME AND PASSWORD PAIR.
   if ($numberOfResults == 1) {
      return true;
   } else {
      return false;
   }
} //end Authenticate


if (isset($_GET['action'])) {
   $action = $_GET['action']; //a little short-hand
} else { //no action specified. Return an error
   header('HTTP/1.1 403 Forbidden');
}

if ($action == 'getListOfFiles') {
   //This code is up to you. You can either iterate through the filesystem, or store this
   //information in the user database
   //File iteration, in the long run, will be more flexible, but will be harder to code. If more than
   //one user share the same folder, that is if you put all user images in
   ///var/www/secureFolder then you can't iterate through the filesystem
   //Assuming we have our file list, return it in json format, or whatever format you prefer.
   //At the very list, we will need to get the folder path for the user

   //IMPORTANT NOTE: How you handle authentication here is important. I don't use sessions,
   //but that is a matter of personal preference. Instead, I authenticate each request.
   if (Authenticate($_GET['username'], $_GET['password']) === true) {
   //these variables depend on YOUR configuration
      $con = mysql_connect("localhost","databaseuser","databaseuserpassword");
      mysql_select_db("my_db", $con); //my_db is the scheme (in MySql) that you are using to store the user information table
      $result = mysql_query("SELECT secureFolder FROM Users where username = \"" . addslashes($_GET['username']) . "\" LIMIT 1" );
      //we could check password again, but calculating MD5 takes CPU...

      while($row = mysql_fetch_assoc($result)) {
         $filePath = $row['secureFolder'];
      }

      //use $filePath to get your $fileList. Again, this is up to you, but in this example I
      //assume that fileList is an array of filenames
      //use echo instead of return, for AJAX requests, to push them back to the browser
      echo json_encode($fileList);
   } else {
      //the user didn't authenticate. Chose how you will handle this
         echo 'NULL';
      }
   }

if ($action =='getFile') {
   //This is the guts of our delivery system. There is a lot you can do here, but this code will show just how to retrieve PNGs from the server and return them.
   //You should be checking file types and such, but for the purpose of this example we will just assume that everything comes to us in the format that we expect (which is not a safe assumption)
   //Also, there is a lot of room for performance improvements here
   if (Authenticate($_GET['username'], $_GET['password']) === true) {
      $con = mysql_connect("localhost","databaseuser","databaseuserpassword"); //these variables depend on YOUR configuration
      mysql_select_db("my_db", $con); //my_db is the scheme (in MySql) that you are using to store the user information table
      $result = mysql_query("SELECT secureFolder FROM Users where username = \"" . addslashes($_GET['username']) . "\" LIMIT 1" ); //we could check password again, but calculating MD5 takes CPU...

      while($row = mysql_fetch_assoc($result)) {
         $filePath = $row['secureFolder'];
      }
      if ( file_exists($filePath . $_GET['fileName'])) {
         $im = imagecreatefrompng($filePath . $_GET['fileName']);
         header('Content-type: image/png');
         imagepng($im);
         imagedestroy($im);
      }
   }
}
?>


Note: If you don't need any image manipulation at all (such as converting from one format to another), then it is faster to use file_get_contents() instead of the GD commands.

4) Create the HTML code
I'll leave the guts of it up to you, but here is the bare minimum (and yes, page encode and doctype ARE BARE MINIMUM). We are also going to include Jquery for our AJAX functions, and another script for our code. I prefer a local copy of jquery, but most people recommend using an online version for caching reasons. The choice is yours.


<doctype html>
<html>
   <head>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
      <script type="text/javascript" src="js/jquery-1.4.3.min.js"></script>
      <script type="text/javascript" src="js/image-loader.js"></script>
   <head
   <body>
      <!-- create a space to place our dynamically loaded images --!>
      <div id='image-area'> </div>
   </body>
</html>


5) Create our javascript code

//we will have our AJAX fire when the DOM is finished loading
//We assume that somewhere in the code you have a username and password. I'm not going to go into the details of this, but you could use cookies or javascript variables, depending on your needs.
$(document).ready(function() {
   $.get('php/Services.php',
      {'action' : 'getListOfFiles', 'user' : username, 'password' : password},
      function(data) {
      //This should all look familiar if you have used Jquery before.
      //This code doesn't have any error checking,
      //but you should ALWAYS check the data coming back
      //from the server to make sure it is safe and isn't null
      for (var k=0; k < data.length; k++) {
         //for each file in the list, we will
         //create a new image element with
         //Jquery and append it to our image-area in our DOM
         $('#image-area').append('<img src="php/Services.php?action=getFile&fileName=' + data[k] + '&username=user&password=pass" />');
      }
   }, 'json') //specify what format we want the data back in
})


That's it. The Jquery code will fetch a list of files, then use that list of files to dynamically generate images on the page. Some notes on the AJAX code: You can use POST to retrieve the file list, but you cannot use POST to retrieve the actual files. The is because of the way in which the underlying XMLHttpRequest handles binary data. Just trust me on this one.

This tutorial leaves a lot for you to flesh out, but it should be a good starting point for the concepts.

Wednesday, December 1, 2010

Updated Theme

As you can see, I no longer have that cool retro Mac theme. Unfortunately, that theme linked to several images in a photo bucket account that seems to no longer be working. Figured I would go with something a little less image heavy.

Rest in peace, you awesome theme.

Friday, October 1, 2010

Samba errors using VFS Modules

I will keep this one short. After struggling for about 30 minutes to get the full_audit VFS module working for Samba, I realized what I was doing wrong. From the looks of others on google, this is a common error that people often don't understand the solution for.

Symptoms:
Samba loads correctly after adding VFS module but shares are not longer accessible
Testparm shows no errors

Errors may include:

log.smbd
Can't find a vfs module [/usr/lib/samba/vfs/full_audit.so]
smbd_vfs_init: vfs_init_custom failed for /usr/lib/samba/vfs/full_audit.so

Syslog:
smbd[7121]: Error loading module '/usr/lib/samba/vfs/full_audit.so': /usr/lib/samba/vfs/full_audit.so: failed to map segment from shared object: Permission denied


Solution:
Check to see if apparmor is running using one of the following (both require root privileges):

/etc/init.d/apparmor status

(or on systems running Upstart)
sudo service apparmor status


If apparmor is loaded, look to see if smbd is set to Enforce. If so, this might be where your problem is. To fix, modify the apparmor profile to allow access to the Samba VFS modules. One Ubuntu 10.04 (and on most systems), they can be found in
/usr/lib/samba/vfs/


An entry in your apparmor profile might looks something like

/usr/lib/samba/** rwkix,


I hope that helps someone out there.

Monday, August 16, 2010

Objects in Arrays

I've been working on a project for work this past when I ran into a silly little mistake that I'm sure others have probably had. I will show you the relevant code (prior to fixing) so you can see where the mistake lies:


//this.IDCounter keeps track of of an index I use to
//generate unique ID's for DOM elements
//this.lookupAdLayout returns a function that I call to
//generate the correct DOM element
this.Create = function(adType) {
   var factory = this.lookupAdLayout(adType);
   this.ads.push(factory(this.IDCounter));
   this.IDCounter++;
   return this.IDCounter-1;
}


//this.ads, as you see in the above code, is an array of custom
//objects. Here I am removing the top element from that array
//and calling the method 'ClearDom' on it.
this.Remove = function() {
   var piece = this.ads.pop();
   piece.ClearDom();
   this.IDCounter--;
}


So, did you catch the mistake? Yeah, neither did I. The problem was each time I would call remove, the array size would shrink, but ClearDom() was always being called on the last created object instead of the top-most object in the array. Here, this is what I mean:

Let's assume the array has 3 objects in it.
this.ads = [obj1, obj2, obj3];

Now, I call this.Remove:
First Time:
   this.ads.length = 2
   piece.ClearDom() gets called on obj3

Second Time:
   this.ads.length =1
   piece.ClearDom() still gets called on obj3

Third Time
   this.ads.length = 0
   piece.ClearDom() STILL gets called on obj3....

Still stumped? Good, so was I. The solution is ridiculously simple once you consider what this.Create() is actually doing. Here, let's break it down:

1) var factory = this.lookupAdLayout(adType); - By calling this.lookupAdLayout, I am returning an object, which in javascript is just a fancy function that looks like this: this.SomeObject = function(). The important point is that what is returned is still a function

2) this.ads.push(factory(this.IDCounter)); - This line can be a bit confusing without seeing the rest of code, but what I am doing is calling the function that was returned in the previous line of code and putting the returned value into the array. In this case, the returned value is a reference to the object I created. IDCounter is used in the constructor and doesn't relate to this problem...

OK, I've dragged it out long enough, and now it's time for the big reveal. Line one returns an object (which is just a function), and functions are passed by reference into other functions (array.push being the function, in this case)... This matters here because each time I call this.Create what I am doing is modifying the variable 'factory' and since the rest of the array only contains references, they all update to match the newly modified 'factory'.Change the second line of this.Create to
this.ads.push(new factory(this.IDCounter));


By using the 'new' keyword, we create a new instance of factory and put that instance in the array, thereby avoiding the reference problem. Like I said, a silly mistake. Hopefully this helps somebody else.

Monday, July 12, 2010

Enums in Unity3d

So, I've seen a lot of hacks getting enums to work properly in Unity. The problem comes when you are trying to pass enums back and forth between scripts. Some say enums are global while others claim you have to reference the script first (somescript.enumName). I've found some of the hacks helpful, but I recently ran into an issue that none of them seemed to fix.

What if you have two instances of the same script that are each trying to reference each other's enums. Here's an example

projectile.js :
enum projectileType = {laser, missle, mine}
var theType : projectileType.laser;

onCollisionEnter(other : Collision) {
if (other.gameObject.GetComponent(projectile).theType == projectileType.laser) {
//do something
}
}


This never seems to work for me, and many of the hacks fail when I add '#pragma strict' to the code. For some reason, the compiler thinks that other.gameObject.GetComponent(projectile).projectileType is of type System.object, and therefor cannot be compared to projectileType. If you are having this problem to, try this:

onCollisionEnter(other : Collision) {
if (other.gameObject.GetComponent(projectile).theType.ToString() == "laser") {
//do something
}
}


Obviously not ideal, but it worked for me. Let me know if it works for you.

Tuesday, June 15, 2010

Accessing C# Scripts From Javascript In Unity

There appear to be several tutorials on going the other way, but I wasn't able to find the solution I needed. With a little tinkering, I found this to work (pretty straightforward really):

If the C# script you are trying to access is called Foo

var bar = gameObject.GetComponent("Foo");

Quotation marks. That's it. HOWEVER, what if we are using #pragma strict, or making this for iPhone? The above code will most likely create problems since either of the situations requires everything to be explicitly typed. So, if Foo where a javascript file, you could do something like:

var bar : Foo = gameObject.GetComponent(Foo);

Pretty simple. Now, try that with a C# file. No luck? That is because of the compilation order that Unity uses. Trying to use "Foo" for your type won't work either. To fix this, you will need to force Unity to compile the C# file first. This can be done by putting the C# file in the folders "Standard Assets", "Pro Standard Assets" or "Plugins." There are, of course, some caveats of doing that. More information Here

So, once you have moved your C# file, you should be able to declare your 'bar' variable as follows:

var bar : Foo = gameObject.GetComponent("Foo");

#pragma strict creates some other interesting quicks that you might not be expecting. More in the next blog post about that.

Monday, April 19, 2010

Unity Built-in Array Woes

Like any sane programmer, I try to keep a reasonable memory footprint for anything I'm coding. So, like any sane programmer, I write everything in Assembly. No, but seriously, poor performance can ruin any good project.

I've recently begun working on a small game using the Unity game engine. One thing I've noticed is that some of the documentation seems to be a bit behind the latest version of the engine (at least for the Javascript function). One great example being the lightning fast built-in array type.

If you are doing anything that targets the iPhone or lower-end hardware, javascript arrays and hashtables can cripple your program. So, like any responsible programmer, you looked up Which Kind of Array or Collection Should I Use?, tried the built-in type for your Javascript:
var enemies = GameObject[16];

and got an error similar to:
Type 'System.Type' does not support slicing.

I've tried every version of that, but no luck. This *seems* to work:
var enemies : System.Array = System.Array.CreateInstance( GameObject, 16 );

Not quit as elegant, but it works.
Please let me know if any of you figured out a simpler way.

Eureka!

So, I've been trying to figure out what to do with this blog space. Originally, it was a place where I was supposed to make up absurd conspiracy theories under some ridiculous alter-ego. Truth be told, it was fun for a couple of weeks, but I just don't have time anymore, and have lost most of those posts anyway.

The other great challenge is that I already have wedemic.net (not sure that's even active anymore), and I contribute to theblogoftomorrow.com, but I'm thinking it may be time to consolidate down to one place where I can ramble on about various programming techniques I discover.

So, if you are looking for odds-and-ends regarding PHP, SOAP, AJAX, Javascript, Apache/WebDav, and various Linux server tricks, hopefully this will prove a valuable resource.

Feedback is alway welcome. Well, unless you are trying to sell "russian viagra love long time," because I'm not interested. I buy that in bulk, and have plenty to last me through the next one hundred years..... I digress.

Thursday, February 4, 2010

Back online

Once again, we are back online. More to follow.