Scientific Software and UI Design

June 27th, 2008

A couple of articles came out recently regarding the quality of good UI design and since this is an area that I’m relatively passionate about, I couldn’t resist adding my voice to the mix.

Last year I did a presentation for a course I was taking on the state of user interface design in the realm of scientific software. My study was based on two papers:

Nielsen, J. and Landauer, T. K. 1993. A mathematical model of the finding of usability problems.1
Javahery, H., Seffah, A., and Radhakrishnan, T. 2004. Beyond power: making bioinformatics tools user-centered.2

To start out I asked those present to look at the user interface of a relatively popular and extremely powerful piece of software used for molecular docking. (which shall remain nameless). Those in attendance were relatively familiar with the subject matter that this software dealt with, and at least one was an expert in the field. I asked them how they would go about performing a few relatively simple tasks. No one could figure it out. The controls were non-intuitive and the interface didn’t adhere to any standard UI design principles.

I then showed them the interface to a relatively popular photo management program:

iPhotoUI.png

I then asked them how they would go about editing a photo or sending a photo to someone via email. Of course the answers where obvious to everyone. Now is this because the Apple engineers were geniuses and the scientific software developers weren’t? Of course not. You don’t write extremely powerful and useful scientific software by being dumb.

UI Design Research

In their research, Nielsen (considered by many to be the expert in usability testing), and Landauer found that you only need to test your user interface design with 5 users to find over 75% of usability problems. Just 5 users! So if it so easy to find problems in UI design, why do we still have such lousy user interfaces in scientific software? Nielsen and Landauer found that a simple model could quantify the cost to benefits ratio of usability testing. The potential payoffs are substantial:

cbRatio.png

The model basically figures in two factors:

  1. The cost of each test user
  2. The savings to be realized by improved usability (i.e. lower support costs, fewer incremental releases to fix UI problems, etc…)

Unfortunately this model breaks down in the realm of scientific software. Here is my hypothesis as to why that is and why it’s likely to continue if nothing changes:

Lee’s Scientific Software UI Hypothesis

A large portion of scientific tools have poor user interfaces due to a combination of the following factors:

  1. There is usually no funding for user interface testing.
  2. The fact is that most funding organizations aren’t interested in paying for usability testing. Consequently it doesn’t get done.

  3. Tool-based papers aren’t evaluated for good user interface design during peer review.
  4. Tool based papers (like other scientific papers) are evaluated for merit and accuracy (as they should be). They are not however evaluated based on UI design principles. Changing this alone would stop the deluge of poorly designed user interfaces in scientific software.

    (For those of you who aren’t familiar with this process, scientific papers (unlike highly-opinionated blog entries like this one) go through a process called peer-review where they are evaluated for their merit and accuracy. This (usually) helps to filter out work of lesser quality.)

  5. Most people writing these software tools either don’t know what good user interface design looks like or don’t care.
  6. In my experience I have found that there are usually two types of people designing scientific software in the life sciences field. CS grads who have lots of training in designing algorithms but usually no training in UI design, or life scientists who have neither.

    This may be an over generalization, but in my experience (and I’m a CS grad), CS grads don’t know how to design good user interfaces. It’s not our fault, these things simply aren’t taught in most CS curriculums. We can optimize O(logn^2) algorithms in our sleep but we often seem to have trouble with basic UI design principles. Either we don’t know about them or we don’t care enough to use them.

  7. There are no savings to be gained in improving the user interface.
  8. This is the big one. There is exactly one payoff for most people in the realm of scientific software (aside from altruistic ones of course), and that is publications. If I publish a paper based on a software tool with a really good and well tested UI design, it counts just as much for me as if I publish a paper based on a software tool designed by a bunch of blind monkeys. (No offense to blind monkeys intended).

Conclusion

So despite the fact that studies (peer reviewed studies mind you) have shown that only a very small number of testers are required to find usability problems, the cost to benefits ratio for scientific tools developers is still unfortunately too high to justify real user interface testing and until that changes we’re going to continue to be stuck with the quality of user interfaces that are currently typical of scientific tools.

  1. Nielsen, J. and Landauer, T. K. 1993. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT ‘93 and CHI ‘93 Conference on Human Factors in Computing Systems (Amsterdam, The Netherlands, April 24 - 29, 1993). CHI ‘93. ACM, New York, NY, 206-213
  2. Javahery, H., Seffah, A., and Radhakrishnan, T. 2004. Beyond power: making bioinformatics tools user-centered. Commun. ACM 47, 11 (Nov. 2004), 58-63

Special Effects with Acorn Part 1: The Product Fade Effect

June 26th, 2008

Acorn is an awesome image editing program from Gus Mueller at Flying Meat. The reason that I like it so much is that it is simple to use, but is still very powerful. It has all of the features that I need to do everything that I would would do with Photoshop, and I don’t have to pay ridiculous amounts of money to buy it.

To show just how easy it is to use Acorn, I’m writing up a series of tutorials that will show how to do some of the common things that people do in Photoshop, but using Acorn.

The Product Fade Effect

The first effect that I’m going to walk you through is the product fade effect. Everyone loves the product fade effect. There’s just something about seeing a beautifully engineered product sitting on a reflective surface that just makes you want to buy it. Our goal is going to be to create this image:

fade_final.png

Doesn’t that fade at the bottom of the image just make you want to go out and buy an iPhone?

So let’s see how you can create equally tantalizing product images.

Step 1 - Load the Product Image

First open Acorn and create a new Document [File->New or Command-N]

step1.png

You can set your image dimensions to whatever you want. I always set mine a bit larger than I need and then crop it down to the right size later. It’s important that whatever you select for your background color is going to work with your product image. The product image that I’m using has a white background, so I set my document background image to white as well.

Once your document has been created, open your product image file [File->Open or Command-O].

step2.png

Select the document with the product image and copy the image [Edit->Copy or Command-C] and then select your new document and paste in the product image [Edit->Paste or Command-V]. Acorn automatically puts the pasted image into its own layer.

step3.png

Step 2 - Position the Product Image

Next we need to move the product image up towards the top. You may not need to do this depending on the size of your image. This is easily done using the Move tool.

Click on the product image to select its layer and then click the Move tool or press V.

step4.png

Click and drag on the product to move it towards the top of the image. This will give you space at the bottom for the faded reflection.

step5.png

Step 3 - Create the Reflection Image

Now we want to create the copy of the product image that we will use for the reflection. There are a couple of ways to do this, and how you do it will depend on what your product image looks like. My product image has a large white background that I don’t want my reflection to have. So I need a way to select my product image independent of its background. One easy method to do that is to use the Magic Selection tool. If your product image doesn’t have a background, you can skip this part.

Click on the product image to select its layer and then click the Magic Selection tool or press W.

step8.png

The “Wand tolerance” setting tells Acorn how sensitive to color it needs to be in making the selection. The higher the number, the less sensitive the tool is to slight changes of color. Here I’ve set my “Wand tolerance” setting to 100. Your value will differ depending on how different your product image color is from its background.

Click on the background of the product image. The magic wand tool should select everything but your product, the little dancing dashes will tell you what you have selected. (Again you may need to experiment with the Wand tolerance setting. If your selection isn’t right and you want to try again, you can get rid of your selection using [Select->Clear Selection or Command-D]).

step9.png

Now this is the exact opposite of what we want to select. We have selected everything but our product and what we want is to select only our product. So next we need to invert the selection [Select->Inverse or Command-Shift-I]. The astute reader may wonder why we didn’t just select the product in the first place. The reason is that most of the time your product image is on a solid background color. Selecting a solid background color can usually be done in a single step with the magic selection tool, whereas selecting the product directly usually takes multiple steps using one or more of the other selection tools. This takes a lot longer and can drive you insane.

Once we have the product selected we want to make a new layer with just our selection [Layer->New Layer With Selection or Command-J].

step10.png

Step 4 - Position the Reflection Image

Now we want to move the reflection image into position. The first thing we do is flip it upside down. Select the reflection layer that you just created and then flip it vertically [Layer->Rotate->Flip Layer Vertical].

step11.png

Now move your upside down product image down to directly underneath the original product image using the move tool. You want the two images to line up just right.

One way to insure that lay line up is to move the image using the arrow keys by clicking the image and pressing down down down… a zillion times until it is in just the right spot. A faster way to do this is to use the handy grid feature [Image->Toggle Grid or Command-'], (you can adjust the grid size in [Image->Grid Settings...]) which then makes it easier to line up the layer using the mouse.

step12.png

Step 5 - Fade the Reflection

This is where the real power of Acorn comes into play, it’s use of filters. There are just a ton of filters that you can apply to your image. We’re only going to need one to create our product fade effect though.

Select you reflection layer (the upside down one) by clicking on it and create a Linear Gradient Filter [Filter->Gradient->Linear Gradient...]. The Apply Filters window appears with the Linear Gradient Filter set to its default settings. These settings will obviously need to be changed.

step13.png

If you mouse over the image preview window you will see two orange points appear, these are the points that define how the gradient is spread out. The way the linear gradient filter works is by drawing an imaginary line between gradient point 1 and gradient point 2 that extends on to the ends of your image. Everything before point 1 it paints color 1. Everything after point 2 it paints color 2. Everything in between the the two points is a subtle gradient between color 1 and color 2.

window1.png

To get our desired effect, we’re going to set color 2 to be completely transparent. To do this, move the little opacity slider next to Color 2 all the way to the left. This will cause the gradient between the two points to be a blend of color 1 (which I leave white here, you will want to set to be whatever your background color is) and transparent.

Next you need to move the gradient points into the right positions. Click and drag the points so that point 2 is right at the top of your reflection image and point 1 is directly under point 2 at the position where you want the fade to be complete. You can also use the text boxes on the right to enter the coordinates of the points directly.

window2.png

When you have your points lined up just right, click “Apply” to see the final result.

step15.png

Conclusion

So in just 5 steps using Acorn you were able to create your very own product fade image that you can use to entice thousands and thousands of buyers.

Great article on software piracy

March 27th, 2008

Several others have mentioned it already, but Brad Wardell wrote a great article about game development and piracy.

This is my favorite quote:

The reason why we don’t put CD copy protection on our games isn’t because we’re nice guys. We do it because the people who actually buy games don’t like to mess with it. Our customers make the rules, not the pirates. Pirates don’t count. We know our customers could pirate our games if they want but choose to support our efforts. So we return the favor - we make the games they want and deliver them how they want it. This is also known as operating like every other industry outside the PC game industry.

Software Company for Sale…Sold!

March 19th, 2008

As you may have already heard, Decimus Software has acquired Polarian.

This sale has worked out wonderfully. Benjamin and his team at Decimus are a really amazing group of people to work with. I am grateful for all of the inquiries that I received and apologize to anyone that I couldn’t get back to personally.

Selling was difficult for us for a lot of reasons, starting Polarian from nothing and bringing it to where it was has been an amazingly fun process. I have learned so much, about software development and business in general.

One of the best things about this deal though, is the fact that Decimus already has a solid reputation in the mac community. Knowing that the group buying Polarian had experience not only programming great mac software, but also running a successful customer-focused business made the decision much easier.

In case you didn’t read the press release, Decimus is having an introductory “50/50″ promotion to celebrate the acquisition. Through the 23rd of March, you can use the coupon code FIFTYFIFTY to purchase Screen Mimic for $50. Even cooler, is that owners of competing products can use the coupon code FIFTYFIFTYUPG to get a competitive upgrade discount of 50%.

Software Company for Sale

February 5th, 2008

Update++

Good news to anyone still interested, our potential buyer has had to back out, so I’m once again considering offers. leefalin@polarian.com

Update

I’d like to thank everyone who showed an interest in Polarian. At this time we have found a buyer and are no longer considering other offers. Thanks again to all those who inquired.

The Reason

When I started Polarian back in 2005, it was with the hope that one day Polarian would be the sole source of income for our family. As 2006 rolled around, Polarian was not yet fulfilling that dream and we made the decision to return to graduate school and pursue my other lifelong dream of being a professor and teaching at the university level.

Throughout 2006 my research still hadn’t really picked up speed and so I had quite a bit of free time. I continued to devote some of that time to further development of Screen Mimic. In 2007 this development came to fruition as Screen Mimic 2.0 was released. From there Polarian’s revenue grew exponentially and went from being a hobby income to something that could in fact fully sustain our family. We started making plans for Screen Mimic 3.0 as well as a new software application that I believed would do even better than Screen Mimic.

During that same period of growth, my research project was also finalized and became much more demanding. As 2008 started it was clear that we could no longer pursue both dreams, so it came time to choose between them. From the title of this entry it should be obvious which one we chose.

Making this choice was difficult for many reasons. We started Polarian from scratch, a blank screen and an idea. We actually began as graphic design company and started writing Screen Mimic after I couldn’t find a screen capture application for the mac that output to Flash format. Since I was in fact a lousy graphic designer, we shifted our efforts fully to Screen Mimic’s development.

After countless hours of hard work we released Screen Mimic 1.0, then 1.2, 1.5, 2.0, etc… Our sales were so high that it no longer made sense to use a third party payment processor so we even developed our own online store.

It has been an amazing process and I have learned so much by it. The Mac developer community is an amazing group of people and I am thankful for all of the support and knowledge that I have gained from them.

The Details

So what does that mean for you? Well it means that you, my faithful reader, now have the opportunity to purchase a complete turnkey software company that is currently earning substantial revenue.

The sale includes:

  • All of the source code for Screen Mimic and all rights pertaining to it.
  • An online store setup to take payments via credit card and PayPal, (though you’ll need to set up your own merchant account with your bank if you want to use it, and your bank would have to use the YourPay API). This isn’t necessary of course because Screen Mimic is setup to use license keys generated by Kagi as well.
  • The source code to the fledgling Screen Mimic 3.0, along with my insider thoughts of where I think that the screencast market is heading.
  • The source code to another software product that is 50% complete which I think will do even better than Screen Mimic.
  • The professional designed icon for the above unnamed software product that was designed by the talented folks over at The Icon Factory

Once you own the code you can of course do whatever you want with it, it will be yours. I’m happy to provide on-going email support to answer any technical questions related to the code, but I won’t be available to do any actual development work.

I’m only interested in cash, not royalties, not stock, nor any other .com type incentives. If you are interested, send an email to leefalin@polarian.com and we can go over the details.

So what happens if I don’t end up selling? I’m not sure. One idea is that I could open source Screen Mimic and see where the community takes it, but I’d rather sell it to someone who is passionate about Mac development.

I’d like to thank…

Throughout this article as I refer to “we”, I mean my wife and I. My wife has been a wonderful strategic advisor, companion, and friend. Anytime I have suggested a bad idea my wife has talked me out if it, and anytime I have suggested a good idea she has encouraged it. There would be no Polarian without my wonderful wife.

Thankfully this isn’t the end for me in the developer community, my research project is such that I’ll still be doing active Cocoa development, just in the scientific/academic community rather than the small business one. Maybe one day when school is finished I’ll return to the exciting and rewarding world that is mac small business.

But for now I want to thank all of the other mac developers that have made this such a rewarding experience for me, I started to name all of you, but it is just too long of a list and I’m sure to leave someone out. I also need to thank all of our customers, all of you that have made Polarian a success.

Let me end by saying again that this has been a wonderful experience filled not just with time spent coding, but time spent interacting with lots of great people. Thank you again.

Processing Your Own Online Payments - A Brief Tutorial [Part 3]

January 17th, 2008

This is the third part of an ongoing series on setting up your own online store.

  1. Part 1: Client-Side Storefront
  2. Part 2: Server-Side Validation and Routing
  3. Part 3: Server-Side Payment Processing
  4. Part 4: Adding PayPal Support

Once again, I’m using a PHP/JavaScript system because that’s what I’m familiar with. If your webhost supports Ruby on Rails, then you’ll save a lot of time by downloading the Potion Store from the folks over at Potion Factory.

And once again a discalaimer:

Disclaimer: I am not an accountant or lawyer, before doing anything involving other people’s money you should check with a lawyer and an accountant.

I’d like to thank everyone that gave me feedback from the previous article, please continue to do so.

Site Map

At the end of this series, I will have described a store that looks more or less like this:

storeMap.png

  1. store.php

    This is the page someone comes to when they have chosen to pay with their own credit card.

    As I mentioned last time, the store.php page now contains everything that we moved out of our original storefront that we need to process the credit card transaction.

    Most of this is just plain HTML forms stuff which we went over in part 1, so I won’t go over it again here.

    One thing that I want to highlight again however is the use of the PHP session_start command at the start of this (and every php page):

    <?php session_start(); ?>

    Once this command has been called, I can now retrieve any value that was saved as a session variable. For example, here is how I retrieve the total price of the order and format it to two decimals places:

    <?=number_format($_SESSION['amountTotal'],2) ?>

    Once the customer information is entered into this page, client side validation is done (which we covered in part 1), the form is submitted in the standard way to confirm.php

  2. confirm.php

    The confirm.php page is where the customer has the chance to validate all of their payment information. It also allows us to get things setup for the final processing of payment information. Here is the relevant code from confirm.php:

    1. <?php
    2. session_start();
    3. require_once ‘./include/storeSettings.php;
    4. if($_SESSION['paymentMethod'] == “cc”)
    5. {
    6.     // Store the customer information in session variables except for the sensitive data
    7.     $_SESSION['firstName'] = $_POST["FirstName"];
    8.     $_SESSION['lastName'] = $_POST["LastName"];
    9.     $_SESSION['company'] = $_POST["Company"];
    10.     $_SESSION['address1'] = $_POST["Street1"];
    11.     $_SESSION['address2'] = $_POST["Street2"];
    12.     $_SESSION['city'] = $_POST["City"];
    13.     $_SESSION['state'] = $_POST["State"];
    14.     $_SESSION['country'] = $_POST["Country"];
    15.     $_SESSION['zip'] = $_POST["ZipCode"];
    16.     $_SESSION['paymentSource'] = “************” . substr($_POST["CreditCardNumber"], (strlen($_POST["CreditCardNumber"]) - 4), 4);
    17.     $_SESSION['email'] = $_POST["Email"];
    18.     // Encrypt sensitive data before putting it into session variables
    19.     $_SESSION['S1'] = encryptString($_POST["CreditCardNumber"]);
    20.     $_SESSION['S2'] = encryptString($_POST["CVV2"]);
    21.     $_SESSION['S3'] = encryptString($_POST["CCMonth"]);
    22.     $_SESSION['S4'] = encryptString($_POST["CCYear"]);
    23.     $payButtonText = “Complete Purchase”;
    24. }
    25. else
    26. {
    27.     .
    28.     // Pay Pal stuff coming in Part 4
    29.     .
    30. }
    31. ?>
    32. <html>
    33. <head>
    34. .
    35. .
    36. .
    37. <form name=’form1′ method=’post’ action=’process.php’>
    38.     .
    39.     .
    40.     .
    41.     <input class=’storePayButton’ type=’submit’ value=’<?=$payButtonText ?>’>
    42. </form>

    On line 4 we check to see if we are processing the credit card ourselves (as opposed to PayPal, which we’ll discuss in Part 4). Then on lines 6 - 22 we proceed to gather up the customer data from the form posting and store them into session variables.

    There are two important security related things to notice here. First of all on line 16 we are storing the credit card information that we display back to the user for confirmation. Notice that we don’t use the whole credit card number, just the last 4 digits. Credit card information should never be shown in full on the confirmation screen.

    Lines 18 - 22 deal with another security issue. Since we obviously still need to have the customer’s credit card number to process the payment, and since that processing doesn’t happen until after the confirmation page, we need a way to pass the credit card information to our processing page. Now, since this is very sensitive information we have to be very careful with how we deal with it. Here is the line of thinking that led to the above solution:

    • Store it in a hidden field on a form - This is only one degree of separation from displaying it in the browser to the user, and is therefore too insecure.
    • Store it in our database and delete it later - Bad idea. Not anywhere in this process will I store anyone’s payment information in a database and neither should you. Even if you plan on removing it at the end of the transaction. As a rule of thumb, credit card information and databases should never go together.
    • Store it in a session variable - This is pretty secure if we’re using an ssl certificate and then it won’t be shown to the user right? Well almost. If you are the only one using the server, and it is your server, then this could be a moderately good idea. However if you are on any type of shared hosting plan, there is a chance (albeit slim), that some other user could access your session variables.
    • Use encrypted session variables - Use session variables, but encrypt the values before storing them. This is what we are going to do.

    Back in part 2 we talked about our storeSettings.php file where we stored our database connection information. Recall that this is in a password protected directory somewhere outside the web root. In addition to the database connection info, we also have the following functions defined:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    <?php
    .
    .
    .
    // Secret encryption stuff
    define('ENCRYPT_KEY', 'some_really_good_password');
    define('ENCRYPT_ALGORITHM', MCRYPT_BLOWFISH);
    define('ENCRYPT_MODE', MCRYPT_MODE_CFB);
    define('ENCRYPT_RANDOM_SOURCE', MCRYPT_DEV_URANDOM);
    function encryptString($stringToEncrypt)
    {
        $iv = mcrypt_create_iv(mcrypt_get_iv_size(ENCRYPT_ALGORITHM, ENCRYPT_MODE), ENCRYPT_RANDOM_SOURCE);
        return base64_encode($iv . mcrypt_encrypt(ENCRYPT_ALGORITHM, ENCRYPT_KEY, $stringToEncrypt, ENCRYPT_MODE, $iv));
    }
    function decryptString($stringToDecrypt)
    {
        $toDecrypt = base64_decode($stringToDecrypt);
        $ivSize = mcrypt_get_iv_size(ENCRYPT_ALGORITHM, ENCRYPT_MODE);
        $iv = substr($toDecrypt, 0, $ivSize);
        $data = substr($toDecrypt, $ivSize);
        return mcrypt_decrypt(ENCRYPT_ALGORITHM, ENCRYPT_KEY, $data, ENCRYPT_MODE, $iv);
    }
    .
    .
    .
    ?>
    

    On lines 6 - 9 we define some information that we always want our encryption routines to use, like the encryption algorithm and the key used to encrypt and decrypt the string.

    Lines 10 - 14 define the encryptString function, which takes a string and returns the encrypted version.

    Lines 15 - 22 define the decryptString function, which takes an encrypted string and returns the decrypted version.

  3. process.php

    Finally we come to process.php, the page that will actually handle the credit card validation and complete the transaction. There are lots of things going on here, so I’m going to break the code up into chunks.

    1. <?php
    2. session_start();
    3. require_once “./include/StoreSettings.php”;
    4. require_once “./include/lphp.php”;
    5. // Connect to DB
    6. $conn = mysql_connect(DB_HOST, DB_USER, DB_PASS) or header(”Location: ” . ERROR_PAGE . “301″); // Error connectiong to DB
    7. mysql_select_db(DB_NAME);
    8. if($_SESSION['isUpgrading'])
    9. {
    10.     $UpgradeCode = mysql_escape_string($_SESSION['upgradeCode']);
    11.     $query = “UPDATE UpgradeCodes set Used=’1′ WHERE Code = ‘$UpgradeCode’”;
    12.     mysql_query($query) or header(”Location: ” . ERROR_PAGE . “302″); // Error updating upgrade code table
    13. }

    The first interesting thing that we do here in lines 8 - 13 is mark the upgrade code as having been used. This prevents multiple people from using the same upgrade key. Notice that if this is some type of reusable coupon code (such as for MacSanta), you wouldn’t want to invalidate the code in this step. How you handle that depends on what type of coupon code you’re dealing with.

    1. // Sanitize Values
    2. $FirstName = mysql_escape_string($_SESSION['firstName']);
    3. $LastName = mysql_escape_string($_SESSION['lastName']);
    4. .
      .
      .


    5. //Run Query to add the user
    6. $query = “INSERT INTO Customers(FirstName, LastName, Company, Address1, Address2, City, State, Country, Zip, PaymentSource, Email, ChargeTotal, OrderTime) “;
    7. $query .= “VALUES (’$FirstName’, ‘$LastName’, ‘$Company’, ‘$Address1′, ‘$Address2′, ‘$City’, ‘$State’, ‘$Country’, ‘$Zip’, ‘$PaymentSource’, ‘$Email’, ‘$AmountTotal’, NOW())”;
    8. mysql_query($query) or header(”Location: ” . ERROR_PAGE . “303″); // Error creating user record
    9. $customerID = mysql_insert_id();

    Lines 14 - 16 begin sanitizing the values that we are going to insert into our database. This prevents nasty SQL injection attacks from happening. As I mentioned before, if you’re not sure whether magic quotes is enabled on your sever, you need to check by calling the get_magic_quotes function before doing this.

    In lines 18 and 19 we build up our insertion query statement that we use to insert the customer data into our database. Notice again that the credit card information is NOT stored in the database. If you have to store something for payment identification later, only store the last 4 digits of the card number that was used on the verification screen earlier.

    Finally on line 20 we run the query, and make sure that everything worked out ok, then on line 21 we retrieve the customer id (aka the primary key) for the record we just inserted. This is a unique key that refers to that row that was auto-generated by the database. We will use this key to link the records between this table and our products table, which I’ll discuss soon. If you don’t understand why a unique key is important, it may be a good idea to review database normalization guidelines before building your customer database.

    1. $ProductQuantity = mysql_escape_string($_SESSION['productQuantity']);
    2. $ProductPrice = mysql_escape_string($_SESSION['productPrice']);
    3. $UpgradeCode = mysql_escape_string($_SESSION['upgradeCode']);
    4. $query = “INSERT INTO Products(CustomerID, ProductCode, Quantity, Price, UpgradeCode) “;
    5. $query .= “VALUES (’$customerID’, ‘123′, ‘$ProductQuantity’, ‘$ProductPrice’, ‘$UpgradeCode’)”;
    6. mysql_query($query) or header(”Location: ” . ERROR_PAGE . “304″); // Error creating product record

    Here is where the Products table comes in. In lines 22 - 27 we insert a record into the products table containing the customer ID that we retrieved earlier. Why have two separate tables? Well if you have multiple products and one customer buys two different things, this allows you to avoid duplication of information. Notice that our example doesn’t handle multiple products at all except conceptually in the database design. However modifying the example to do that would be relatively trivial for someone with a moderate amount of programming skills.

    1. // Process payment
    2. $paymentApproved = false;
    3. $paymentResponse = “”;
    4. if($_SESSION['paymentMethod'] == “cc”)
    5. {
    6.     $mylphp=new lphp;
    7.     $myorder["host"] = “super.secure.ccprocessor.com”;
    8.     $myorder["port"] = “90210″;
    9.     $myorder["keyfile"] = “./include/myCert.pem”;
    10.     // Credit Card Data
    11.     $myorder["cardnumber"] = decryptString($_SESSION['S1']);
    12.     $myorder["cvmvalue"] = decryptString($_SESSION['S2']);
    13.     $myorder["cardexpmonth"] = decryptString($_SESSION['S3']);
    14.     $myorder["cardexpyear"] = decryptString($_SESSION['S4']);
    15.     $myorder["chargetotal"] = $_SESSION['amountTotal'];
    16.     // Billing Data
    17.     $myorder["name"] = $_SESSION["firstName"] . ” ” . $_SESSION["lastName"];
    18.     $myorder["address1"] = $_SESSION["address1"];
    19. .
      .
      .


    20.     // Send transaciton
    21.     $ccResult = $mylphp->curl_process($myorder);
    22.     $paymentResponse = $ccResult["r_ordernum"] . “|” . $ccResult["r_ref"] . “|” . $ccResult["r_approved"] . “|” . $ccResult["r_code"] . “|” . $ccResult["r_message"] . “|” . $ccResult["r_avs"];
    23.     $paymentApproved = ($ccResult["r_approved"] == “APPROVED”);
    24. }
    25. else
    26. {
    27.     // Pay Pal stuff coming in Part 4
    28. }

    Now begins the actual credit card processing. This section may vary somewhat depending on whom you use for your credit card processing, but if you use LinkPoint like I do, then it could look very similar.

    Lines 33 - 36 begin by instantiating the lphp object provided by the LinkPoint library we included on line 4. This is the object that we will send over a secure connection to LinkPoint. On lines 34 - 36 we provide some basic configuration information, and on lines 44 - 45 (and beyond through the ellipsis) we provide the customer information.

    In lines 38 - 42 we retrieve our encrypted session variables which contain the credit card processing data, decrypt them and assign them to the appropriate keys in our LinkPoint object. (We used the decryptString function which we defined above to do this)

    Finally on line 47 we send the transaction to LinkPoint using the curl_process function, which is defined in the LinkPoint library. This function basically opens a secure connection to the LinkPoint servers, sends the data in the $myorder variable and stores the response in $ccResult.

    Lines 48 and 49 parse the response information. There is a lot of information stored in various keys that may be handy later, but the only thing I really need separate at this point is the approval code, which is retrieved on line 49. Since it may turn out that I do need some other part of the response for something later, I do store the rest of it in the database at a later point in the process.

    1. if($paymentApproved)
    2. {
    3.     // Update Database with CC Approval Results
    4.     $escapedResponse = mysql_real_escape_string($paymentResponse);
    5.     $query = “UPDATE Customers set Approved=’1′,CCResult=’$escapedResponse’ WHERE CustomerID = ‘$customerID’”;
    6.     mysql_query($query);
    7.     // Check CCV Code Results for CC orders
    8.     if($_SESSION['paymentMethod'] == “cc”)
    9.     {
    10.         $CCVResult = substr($result["r_avs"], 3, 1);
    11.         if($CCVResult == “N”)
    12.         {
    13.             // Void bad ccv codes
    14.             $myorder["ordertype"] = “VOID”;
    15.             $myorder["oid"] = $result["r_ordernum"];
    16.             $result = $mylphp->curl_process($myorder);
    17.             $voidResponse = $result["r_ordernum"] . “|” . $result["r_ref"] . “|” . $result["r_approved"] . “|” . $result["r_code"] . “|” . $result["r_message"];
    18.             $query = “UPDATE Users set Voided=’1′,VoidResult=’$voidResponse’, VoidTime=NOW() WHERE OrderID = ‘$orderID’”;
    19.             mysql_query($query);
    20.             // Send to error page with error for bad CVV code
    21.             header(”Location: ” . ERROR_PAGE . “306″);
    22.         }
    23.     }

    Now, here is a subtlety that may not bother you if you aren’t using LinkPoint. Remember that little CCV code (the security code on the back of the card)? It turns out that if everything else about the order checks out, the order will come back approved even if the number the customer enters for their security code is completely bogus. Why is that? Well it is mainly because in some uses of the LinkPoint API the CCV isn’t collected. (Like when the customer swipes their card in a terminal). But since this is a “Card not present” situation, it is very important that you verify the CCV code.

    The first thing we do on lines 57 - 60 is store the entire credit card processing response in the database. Note again, that this does not include any of the credit card information, just the response from the processor.

    On line 62 we make sure that this is a credit card transaction (as opposed to PayPal) before we try and check the CCV code. Next on line 64 we retrieve the character from the avs response code that indicates whether the security code was good. If it wasn’t good, lines 67 - 75 void the transaction both with the credit card processor and in our local database, and then forwards the customer to an appropriate error page.

    If everything checks out ok, we proceed to lines 79 - 81 when generates a unique license key for our customer:

    1.     // Create License Key
    2.     $stringToEncode = generateUniqueAndSecureString();
    3.     $licenseKey = md5($stringToEncode);
    4.     $formattedLicenseKey = “SM21-” . rtrim(chunk_split(strtoupper($licenseKey), 4, “-”), “-”);
    5.     // Update product record with license key
    6.     $query = “UPDATE Products set LicenseKey=’$formattedLicenseKey’ WHERE CustomerID = ‘$customerID’”;
    7.     mysql_query($query);
    8.     // Email results
    9.     $receiptText = “Thanks for your purchase! blah blah blah….”;
    10.     $receiptText .= “More important stuff.”;
    11.     mail($_SESSION["email"], “Your Receipt”, $receiptText, “From:sales@me.com\r\nReply-to:productsupport@me.com\r\nBcc: receipts@me.com”);
    12. }

    I’m using a pretty simplistic license key generation scheme here. It involves three steps:

    1. Come up with a secret unique string
    2. Hash it using MD5
    3. Format it to make it look like a license key

    The security of your license scheme will all come down to how you perform step 1. I’m sure that there are well researched and tested methods for this, but don’t waste too much time on it. As Brent Simmons once said:

    There are two kinds of people in this world, those that pay for software and those that don’t, and never the twain shall meet.

    On lines 83 - 84 we update the products table with the license key we generated for that customer. Notice that despite all of my harping on about database normalization that this code won’t work if you have multiple products. (Unless they all accept the same license key). If you did have multiple products, you would need to loop through lines 78 - 84 for each product you sold in that order.

    Lines 86 - 87 build the string we use for our confirmation receipt. Some people use fancy rich text or html, I just use plain text. However you setup your email receipt, you should include the license code and registration instructions here, this will save you quite a bit of support time later.

    Line 88 actually mails the receipt, notice that the “From” field is a dummy sales email address, while the Reply-To value is set to a product support email address. Also notice that another email address is BCC’d, this allows me to be immediately notified of all of my sales which gives me a warm fuzzy feeling.

    1. else
    2. {
    3.     // Update Database with Failure code
    4.     $query = “UPDATE Customers set Approved=’0′,CCResult=’$ccResponse’ WHERE CustomerID = ‘$customerID’”;
    5.     mysql_query($query);
    6.     header(”Location: ” . ERROR_PAGE . “307″); // Error validating payment
    7. }
    8. ?>

    Finally on lines 90 - 96 we deal with what happens if the payment wasn’t approved by the credit card processor. We simply update the database with the failure code sent back from the processor and redirect the customer to an error page. This failure code is very useful to have when your customer emails you and want to know why you aren’t accepting their card. It is not however an all knowing response that will help you know the exact problem.

    Basically there are three types of responses that you’ll be able to discern. Bad Address, bad CCV code, and rejected. (There is also a fraud response that link point sends but that usually just means the customer tried to process the card again to quickly). Bad address and bad ccv code are pretty self explanatory. The other code is not.

    If you customer contacts you and wants to know why the order was rejected, then all you can do is have them call their issuing bank to find out. It will be a waste of time for you to try and ask your processor, as they won’t tell you anything other than it was rejected. (Nor should they, since the “why” is really none of your business).

    Finally lines 98 - 108 show some sample HTML code that appears to the user upon successful completion of this process:

    1. <!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN” “http://www.w3.org/TR/html4/loose.dtd”>
    2. <html>
    3. <body>
    4. .
      .
      .


    5. <div class=’storeResponseText’>Your order has been approved! Here is your registration information (This information will also be sent to you via the email address you
    6. provided) :</p>
    7. <table border=’0′ class=’licenseTable’>
    8.     <tr><td class=’storeResponseText’>User Name:</td><td class=’licensekey’><?=$_SESSION["firstName"] . ” ” . $_SESSION["lastName"] ?></td></tr>
    9.     <tr><td class=’storeResponseText’>Email Address:</td><td class=’licensekey’><?=strtoupper($_SESSION["email"])?></td></tr>
    10.     <tr><td class=’storeResponseText’>Product Key:</td><td class=’licensekey’><?=$formattedLicenseKey?></td></tr>
    11. </table>
    12. </div>

    I include the same registration information as the email receipt, but you can of course include whatever you want.

Conclusion and Words of Warning

If you think that the above code has a lot of places where things could go wrong, you are right. You need to test every possible failure path that you can think of. This is where having a test plan really comes in handy. Test, test, and retest. Your credit card processor should have a test mode that you can use to process test payments. Use this mode to make sure that things happen the way they should with a bad card number, bad security code, bad upgrade key, etc…

Make sure that if the customer’s name is something like “John O’; DROP TABLE Customers; –’”, that you handle that case correctly. Finally when you switch to live mode, test some more. Test every case again with your own credit card. Don’t worry, you should be able to void your transactions after each test. (If you can’t you should really switch payment processors). Test to make sure that a cleared payment does in fact appear in your bank account. Test everything. Test Test Test. The next day, test some more. Every time you make a change test EVERYTHING again.

If you see an unusually long stretch of time without sales, test again. Just last week we had three days of seemingly no sales, but it turned out that there was just a problem with PHP’s mail setup that our host made which prevented the email from being sent. Fortunately the sales were still stored in the database so we didn’t lose any information.&lt/soapbox>

Finally another word of warning: I am not an accountant or a lawyer, before doing anything involving other people’s money you should check with a lawyer and an accountant.

Don’t quote me on this

November 8th, 2007

Blogs and Twitter are amazing ways to share your thoughts and get feedback. For indie mac developers Twitter is especially nice. Being an indie developer can be a lonely experience sometimes, and Twitter provides a sort of virtual set of coworkers that you can use as a stress reliever. (Not that anyone should think that Twitter or any other online service is a good substitute for interaction with real people). Fraser Speirs wrote about this and summed up the benefits of Twitter nicely with this statement:

Twitter has become a social/professional network for the independent Mac developer…I love Twitter because it has the immediacy of IM but removes the expectation of a prompt reply.

An interesting question related to this ability to shout out to the world as it were, is where does your company’s voice end and your personal voice begin? Especially when you are an indie developer and in a lot of people’s eyes (you == your company).

Earlier today Byran Gardner at Wired quoted something Daniel Jalkut said on Twitter.

Someone raised the question of whether or not Twitter feeds are something that you should ethically quote, especially without permission.

The question really isn’t twitter specific, the question to me is, if you are an indie software developer, where does your company’s voice end and your personal voice begin?

Does it end at the press release, the company blog, your personal blog, your twitter account, your email?

I don’t know the answer, but I imagine that this is an issue (not Daniel’s quote, but the issue in general) that is going to need some analysis.

Translucent Menu Bar Fix a la Automator and Quartz Composer

October 29th, 2007

I haven’t met anyone that likes the transparent menu bar in Leopard.

So here are a couple of ways to fix it:

  1. Use a program to fix it like the Non-Transparent Menu Bar Program from the fine folks at Many Tricks.
  2. Or paint a white rectangle on the top of your background picture the same height as the menu bar.

I personally didn’t want to have a program running in the background all the time just to fix a UI flaw, however I’m one of those people that changes their desktop background fairly often, so I also didn’t want to have to manually edit my background every time it changed.

The Magic of Quartz Composer

Using Quartz Composer it is easy to do things like take a white rectangle and scale it to the appropriate size and location to be exactly on top of some image of arbitrary size. Using the “Image Filter” template sets our composition up to be used by any application that can take advantage of Image Filters.

All we have to do is put our Quartz Composer file in ~/Library/Compositions or /Library/Compositions and its ready to be used.

I still want to be able to apply this directly from the finder though.

Automator to the rescue

Automator has a lot of great ways to automate repetitive tasks. It takes exactly five tasks to do what we want:

  1. Get Selected Items - This will take whatever file you have selected in the finder and pass it to the next task.
  2. Copy Finder Items - Unsurprisingly this task makes a copy of the file from task 1. In our case we’re copying it to the Pictures folder.
  3. Rename Finder Item - This task is amazingly useful, here we’re using it to add an _mb after the filename (but before the extension) of our copied image.
  4. Apply Quartz Composition Filter to Image Files - Here is where the magic happens. In this action the file we renamed in task 3 gets modified by the image filter of our choice.
  5. There are a couple of parameters here that start with _protocolInput… that our composition doesn’t use for anything, so don’t worry about those. Just select the filter to use from the drop down box.

  6. Set the Desktop Picture - Finally using this action we take our newly filtered image and set it as the desktop background.

Save this workflow as a Finder plugin “File->Save as Plug In…” and you can now right click on any image and through the automator sub menu select your newly saved workflow that will do all of the above for you.

I have included the source files and install instructions here. I could have packaged this all up into a nice installer package but I don’t really like installer packages so I didn’t.

Also, this particular setup only works if your desktop resolution is set to 1440 x 990. There doesn’t seem to be a way to deduce the screen resolution from inside automator or Quartz Composer, so you’re stuck with the resolution I use on my MBP. If there is a way to get the resolution, someone tell me and I’ll update the source files.

Seeing i to i with the iPhone

October 23rd, 2007

Just last week I picked up an iPhone. I’ve been very happy with it, and for a 1.0 product it is absolutely amazing, especially from a software point of view.

There are of course a couple of things that I would love to see on it, a to-do list would be nice, some better way to scroll through really long documents, minor things really. Its great how the iPhone syncs with iTunes and iPhoto so well, and the email integration (now that I have my gmail account forwarded through an IMAP server) is great too. Throw in the great integration of NetNewsWire and NewsGator’s iPhone site and nearly all of my typical online tasks are supported.

There are however a couple of integration related things that are puzzling to me. While everyone is clamoring to see their favorite 3rd party apps ported to the iPhone, what I’d like to see is more of Apple’s applications supported by the iPhone.

iPhone doesn’t work with iWork

I’ve been using Pages as my primary word processor since it came out. It does everything that I need a word processor to do. But unfortunately although I can read documents created by Microsoft Word, there is no way (yet) to read documents created in Pages unless I export them to PDF or Word format first. The same goes for Numbers vs Excel. (Though I only currently use numbers for minor things since it doesn’t support two key features that I use in Excel, good scatter plots and regression lines).

What I would really like to see in terms of iWork integration is a way to view Keynote presentations on the iPhone, even better would be the ability to pipe them out through the TV out cable.

Leading by Example

Since the iPhone “SDK” came out, Apple has really been pushing hard for people to develop iPhone-optimized sites. They setup a section on their developer site that helps you develop web apps for the iPhone and have recently added a new section on their site where developers can showcase web applications specifically designed for the iPhone. This is a great idea and it has made it possible for me to find some great sites that I wouldn’t have known about otherwise.

Unfortunately Apple’s iPhone web apps site looks lousy when viewed on the iPhone. When you browse to that site, you have to do a combination of rotating, scrolling, pinching, and tapping just to get the app list to fit on a screen in such a way to see the icon, the description, and the link button at the same time.

The iPhone is still awesome

I can certainly see why if the iPhone dev team had to choose between getting Word and Excel or Pages and Numbers compatibility working in 1.0 that they went with the compatibility with Office. The demand for that feature is certainly much greater. I just hope that one day in the not to distant future, that iPhone and iWork will see i to i.

Screen Mimic 2.2 and Beyond

October 22nd, 2007

Screen Mimic 2.2 has been released!

This version brings a couple of bug fixes, and most importantly, Leopard compatibility. (Or at least as close to Leopard compatibility as can be claimed prior to the 26th).

Daniel Jalkut make a great point that any update related to making your software compatible to Leopard should be free, and as you should be able to tell from the version number, this update is free for all 2.x users.

With Leopard compatibility out of the way I wanted to also provide a brief glimpse into the future of Screen Mimic. This is also your chance to chime in and be an advocate for your favorite missing feature.

Here is a brief glimpse of what is on the future slate for 2.3 (which will also be a free upgrade) which I plan to have out by Nov 30th.

2.3 Roadmap

  1. Resize recording on export - This is something that a lot of people have been asking for, the ability to record at any resolution and then scale it down based on what you want to use the video for.

    One of the key features for a lot of our users is that Screen Mimic is simple to use. You don’t really need to know anything about video codecs or post processing to use it, and in implementing this feature, I am being very careful to try and preserve that simplicity.

  2. Batch encoding - This will allow users to take a recording and encode it using several sets of options, without having to wait for each recording to finish before setting up the next run.
  3. Crash recovery - Sometimes applications crash. I wish it never happened to Screen Mimic, but sometimes it does. Nothing is more frustrating when this happens. One of the main causes of frustration though stems from the fact that unless you saved your recording prior to encoding it, you’re going to lose that data.

    Crash recovery will change that. The way this new feature will work is, if Screen Mimic crashes while encoding a recording, and you relaunch the application, Screen Mimic will automatically detect the fact that it crashed. It will then ask you two questions:

    1. Do you want to send a copy of the crash report to Polarian? - This helps us to find the cause of the crash and try and fix whatever caused it. This is completely voluntary and has no effect on crash recovery, it is just something to help us isolate and fix the bug that caused the crash.
    2. Do you want to recover your last recording? - Answering yes to this question will do two things. First it will create a copy of the recording in the location that you specify. Then it will switch to the encode recording screen so that you can encode the video without having to record it again.

    Crash recovery has two goals. The first and foremost is to protect your recording. The second is to provide Polarian with the crash report that we need to try and prevent the problem from happening again.

3.0 Roadmap - Under construction

Screen Mimic 3.0 is going to bring some amazing changes to the software. We’ve gathered lots of feedback on how people want to be able to use Screen Mimic and in 3.0 we’re reengineering large parts of the software to accommodate those use cases.

I can’t provide detailed features at this time, however I can tell you that 3.0 will be Leopard only. I can also tell you that 3.0 won’t be a free upgrade, however it will be a cheap upgrade. Cheaper than the 1.x to 2.x upgrade fee.

As I said, this is a great time to provide feature requests and suggestions for what you would like to see for Screen Mimic’s future. I can’t promise that we’ll provide everything that everyone wants, but I can promise that we’ll do our best to make Screen Mimic as great as possible for as many people as possible.