The Resplendent Developer

Software Development and Software Quality Assurance

February 10, 2014
by Russell Sinclair
Comments Off

Reading the Cassandra Database with Java

The company I currently work at is predominantly a Linux shop working with Java/Mysql/php/Cassandra. As I become more fluent in these technologies, having come from a strictly Windows/.Net world, I find myself tripping over a number of things that people who work with MS take for granted. The main thing I ran into recently was trying to figure out how to query the Cassandra database with Java. The main problem I ran into was a great lack of really useful or up to date documentation. This post is in hopes of helping muddle through and get started. Hopefully, someone will find this useful rather than having to “read the code” to infer functionality.

Cassandra queries are in the form of “row keys”. As a non-relational database, each row can have a different number/name of columns from the previous/next row. A row key will allow you to pull back one or more rows in a result set to work with.

In this article, we will access Cassandra with the software package called “Hector”. https://github.com/hector-client/hector
One other detail you will notice is the reference “Thrift” while working with Java/Cassandra. Thrift is the legacy API for older clients. It is recommended that going forward, projects use CQL for accessing Cassandra. We are working with Hector which will handle much of the heavy lifting for us.

My example will assume a fairly simple Composite row key and result set. I’ve been working with this type of thing recently, so I have the most familiarity with it.

Row Key

Name Type
varname1 String
varname2 String
varname3 String

Column Names

Name Type
varname1 String
varname2 String
varname3 String

First off, you will need the Hector library. You can obtain this via Maven (I eventually want to talk about the configuration nightmares I’ve had with Eclipse/Spring Tool Suite, but that is a future article).

The libraries I’ve imported are as follows

import me.prettyprint.cassandra.model.ConfigurableConsistencyLevel;
import me.prettyprint.cassandra.serializers.*;
import me.prettyprint.cassandra.service.*;
import me.prettyprint.hector.api.*;
import me.prettyprint.hector.api.beans.*;
import me.prettyprint.hector.api.factory.*;

I could probably get away with fewer imports, but I tend to over import when I prototype knowing that the optimizer SHOULD take care of most of the unused code.

The first thing I’m going to do is put together my various settings that I will need to form the query.

ConfigurableConsistencyLevel consistency = new ConfigurableConsistencyLevel();
consistency.setDefaultReadConsistencyLevel(HConsistencyLevel.valueOf("ONE"));
consistency.setDefaultWriteConsistencyLevel(HConsistencyLevel.valueOf("ONE"));

For those not familiar with Cassandra, consistency levels relate to the configuration of the Cassandra cluster. A cluster can have one or more servers that mirror each other. When a query is executed, all servers have the opportunity to answer (it’s a tad more complex than this, I don’t want to get any further into the weeds). The read consistency is “How many responses do I need that agree until I can return?”. In this case? One.

WriteConsistency is similar but it’s “How many servers have to be written to until I cam move on?”. In this case? One.

Next, we will create a basic configuration object. This is not technically necessary, but when dealing with timeouts, it’s makes life easier. Remember, the Hector default is that if you don’t set a timeout, there isn’t one.
CassandraHostConfigurator cassandraHostConfigurator = new CassandraHostConfigurator("localhost:9160");
cassandraHostConfigurator.setMaxActive(20);
cassandraHostConfigurator.setCassandraThriftSocketTimeout(5000);
cassandraHostConfigurator.setMaxWaitTimeWhenExhausted(5000);

My Cassandra database lives on my local system. If I opened up a SSH tunnel, the “localhost” would still work. Cassandra talks on port 9160.

‘setMaxActive’ method sets the maximum number of active connections to Cassandra.
‘setCassandraThriftSocketTimeout’ method takes the number, in milliseconds, to pass to java.net.Socket#setSoTimeout. (http://en.wikipedia.org/wiki/Unix_domain_socket)
‘setMaxWaitTimeWhenExhausted’ is the global timeout for the Cassandra query.

Now, lets create a cluster interface to talk to!

Cluster myCluster = HFactory.getOrCreateCluster("TestCluster", cassandraHostConfigurator);

Now, when I say “TestCluster” there? You can put anything there. It doesn’t matter. As far as I can tell, it doesn’t get used outside of this line. The second argument is the configurator object I created previously.

Next up, we point the query at the Keyspace. In Cassandra, a keyspace is like a “schema” in a SQL database.

Keyspace ksp = HFactory.createKeyspace("KeyspaceName", myCluster, consistency);

Obviously, you’d replace “KeyspaceName” with whatever keyspace you are working with. In the code I was writing, the keyspace I worked with was called “Analytics”. The second argument, is Cluster object we just created. Finally, we have the consistency settings object.

Now we can start building our query. Unlike working with SQL, you need to know exactly what your data looks like in Cassandra. For example, in MySql/SQLServer when you query, you can specify the datatype when you parse the results – e.g. a field may be a number, but you could manipulate it as a string if you so chose. In Cassandra, you cannot do this. If the value is an Int, you need to receive an Int. We start seeing this here.

MultigetSliceQuery<Composite, Composite, BigDecimal>
multigetSliceQuery = HFactory.createMultigetSliceQuery(ksp,
new CompositeSerializer(),
new CompositeSerializer(),
me.prettyprint.cassandra.serializers.BigDecimalSerializer.get());

OK, You are using the MultiGetSliceQuery class. This is the class that will actually be running the query you specify. You use pass in your keyspace and then your serializers. My row key is of “Composite” type, so that is my first argument. The second field represents the column name structure expected. The last one is the serializer to read the value of the column. You no doubt notice that I had to specifically reference the class. I ran into an issue where there was a conflict with java.math that it tried to pull instead. So, I was forced to dereference in my prototype code.

Now, I can add the column family I want to query. For SQLers, Column Family is like a table.

multigetSliceQuery.setColumnFamily("MyValueTable");

As before, use the name of the column you are querying.

Finally, we can actually start building the parameters of our query. For my cassandra-cli query, I could have a query that looks like this:

get MyValueTable["arg1:arg2:arg3"];

When you are querying Cassandra, and your row key is a Composite (that is a key that is formed by multiple arguments, it could like the above with the “:” character separating them when you type the query. In that case, my Composite row key would be built like this.

List<Composite> keys = new ArrayList<Composite>(); Composite rk = new Composite();
rk.addComponent("arg1", StringSerializer.get());
rk.addComponent("arg2", StringSerializer.get());
rk.addComponent("arg3", StringSerializer.get());
keys.add(rk);

We could actually create a large number of rows keys to query and return multiple result sets. However, we will just stick with the one and

multigetSliceQuery.setKeys(keys);

add it to our query object. Finally, we set the range we want to return. We want to pull the entire dataset, but we could choose smaller values.

Composite startRange = new Composite();
startRange.add(0, new Long(0));
Composite endRange = new Composite();
endRange.add(0, Long.MAX_VALUE);
multigetSliceQuery.setRange(startRange, endRange, false, Integer.MAX_VALUE);

We set start and end points (format Composite). The third argument indicates whether we want the data to be sent to us in reverse. If we did, we set that to true, but, we’d also reverse the startRange and endRange arguments. Note that for end range, I gave it an arbitrarily large number. I’ve stolen this idea from a coworker for the point of this demo, because it greatly simplifies things. Same for the “count” which is the last item.

We can finally run the query!

QueryResult<Rows<Composite, Composite, BigDecimal>> result = multigetSliceQuery.execute();
Rows<Composite, Composite, BigDecimal> rows = result.get();

“rows” is now a container of Rows of Columns. All that is left, is to pull the data out and show it!

for (Row<Composite, Composite, BigDecimal> row : rows)
{
ColumnSlice<Composite, BigDecimal> cs = row.getColumnSlice();
for(HColumn<Composite, BigDecimal> col : cs.getColumns())
{
BigDecimal value = col.getValue();
System.out.print(col.getName().get(0,StringSerializer.get()));
System.out.print(":");
System.out.print(col.getName().get(1,StringSerializer.get()));
System.out.print(":");
System.out.print(col.getName().get(2,StringSerializer.get()));
System.out.print(" = ");
System.out.print(col.getValue().toString());
System.out.print("\tRecorded At\t"+(new Date(col.getClock())).toString());
System.out.println();
}
}

And we turn to our handy nested, “foreach” loop. Note the structure of the Row container is consistent with the above structure.

Inside the loop, our first command is to break the Row down into a list of columns using the “getColumns” method. I pull each column, one by one, into a “HColumn” object to work with.  In this example, my columns have a Composite name. It’s similar to the row key shown above, 3 string variables. I grab the value and print it out.

All in all, it’s pretty easy. The only difficult part is the uphill battle of figuring out what/why you are supposed to do.

Recommended Reading:
http://www.amazon.com/Cassandra-Definitive-Guide-Eben-Hewitt-ebook/dp/B004FGMTZY/
http://www.datastax.com/sites/default/files/hector-v2-client-doc.pdf
http://hector-client.github.io/hector/source/content/API/core/1.0-1/index.html
http://wiki.apache.org/cassandra/GettingStarted

October 19, 2013
by Russell Sinclair
Comments Off

Aaaaand, we’re back!

At the beginning of 2013, I started a new job which actually took me out of the Windows/.Net/SQL Server world into the Unix/Open Source/MySQL/Cassandra world. It has been extremely exciting and a great deal of fun. However, what this means is that it no longer made sense to run a blog in .Net as I wanted to have something that more reflected what I was doing on a day-to-day basis. So, when my domain renewal came up I made the move from Winhost.com (a decent option for those who wish to run an inexpensive .Net website) to Dreamhost. This also meant that I moved to WordPress.

The thing I will say about Dreamhost, is that it was extremely simple to spin up this blog and to configure it. Furthermore, I had a HUGE amount of options for plugins I could use on the site, including a large amount of anti-spam options that were all free.

As an aside, I never really got the editor to work well using BlogEngine (my previous blog software). As I said, this was very simple and works very well.

Short post this time I know, but I wanted to officially welcome you to the new home!

April 13, 2013
by Russell Sinclair
Comments Off

Configuring a new Windows 8 Machine

This is another general “tech geek” post, but I think there are some useful bits in it!


My wife has recently gone back to graduate school. I had originally given her my old Compaq laptop (circa 2005) to start with. While it has worked well for her modest needs such as Word Processing and running an old version of a piece of statistical software which could run on even OLDER systems, even modest modern apps would bring it to it’s electronic knees. So, much to her irritation and my joy, it was time to get her a new system to work on. Even modest modern hardware requirements would be good enough but it would mean that it might be difficult to get Windows 7. Yes, I’ve used Windows 8 and I think it’s massively painful. We went with a Lenovo U400 series system primarily because if I had to downgrade it, there would be full driver support. However, I decided that it would be best to try to cope with 8 and then downgrade if she couldn’t tolerate it. Thankfully, after my basic setup steps, I managed to configure the system in such a way that it was useful without a touchscreen.

 
My first steps involved my usual system cleanup I run if I don’t want to do a format and reinstall of the OS. This involves a visit to pcdecrapifier.com. This is a nice little tool that lets you kill off a number of applications that PC makers usually bundle in to the system. The main thing I pulled was McCafee anti-virus as Windows 8 already comes with the “Defender” product which kills Malware that is of concern. I’ve long been a fan of Security Essentials and this is the newest iteration of that software. There were a few other trash apps that I knew weren’t needed. 
 
Now I come to the first wrinkle that Windows 8 introduces. The Windows 8 User Interface (formerly known as “Metro UI” until a lawsuit put a stop to that) has a number of applets from the Microsoft App Store (their answer to the Apple App store) that Decrapifier cannot see. While easy to uninstall, it was something I had to do manually. Yes, these little things are nice, but in reality they just took up space. This MASSIVELY reduced the size of the Start Screen and allowed everything to fit on a single page. This meant no more having to try to scroll the page with a mouse. Amongst all the things I removed, I pulled IE off of the Start Screen as well. The “Metro” version of the IE 10 is, quite frankly horrid. It does not behave the same as the “Desktop” version and is is generally difficult to use on a computer. It is really designed for a touchscreen. I honestly cannot recommend it. If a website uses Flash, but isn’t on Microsoft’s “Whitelist” (such as the company I work for), the Metro version of IE will act as if Flash has not been installed.
 
Next, I headed over to ninite.com and created a installer for Chrome, Firefox, VLC, QuickTime, .Net (wasn’t necessary), Silverlight, Java Runtime, Picasa, Dropbox, Evernote, and 7-Zip. I love this tool because it quickly and effortlessly installs the software I need on a system and does it in an “unattended” way so that I don’t need to babysit it. All of these applications placed shortcuts on the Start Menu so that if she needs to run software, the items are there, just like icons on a desktop. When you click on one, the system drops to the classic desktop and runs the application. THIS IS THE KEY TO COPING! It takes the pain of the “Metro UI” away and makes the system extreme simple to use!
 
The last piece of software I installed was Office 2010. I prefer working with ISO files instead of actual CDs, and in this case that was both necessary and easy. While this particular system does not come with an optical drive, Microsoft FINALLY included the ability to mount an ISO natively in much the same way Apple has done for years. I was especially happen about remembering this tidbit because my go to software “MagicISO” would not install. MagicISO touches some drivers in Windows that Microsoft does not want touched. While you can disable that, the “Secure Boot” technology further stops it cold. I probably could have gotten around that, but I decided not to. Of course, what I did instead of mounting the ISO with Windows 8, was to mount the ISO on my laptop, share it and install over my network. 
 
Incidentally, I did one other little bit of configuration. I opened up Word and Excel and changed the default save and recovery directories to Dropbox. That means, should she have a hard drive failure or simple forget her system, she can access anywhere. Furthermore, Dropbox offers version control. She is able to check “previous versions” of documents to help recover from mistakes. I *highly* recommend doing this.
 
In conclusion, my sweetie has a new, light ultrabook that she seems genuinely happy with. Furthermore, I got a chance to figure out how to make Windows 8 bearable!

March 3, 2013
by Russell Sinclair
Comments Off

Upgrading to a Solid-State Drive

This is less programmer/QA and more in the “Tech Geek” realm, but I figured it was worth writing about.

For my birthday this year, I decided that I could do a bit of a hardware upgrade on my main system: Dell XPS L702X laptop. I replaced the main drive with a 256 Gigabyte Solid State Drive and placed the hard disk in the second drive bay for storage. This prompted an interesting journey. For those who don’t know, the beauty of a solid state drive is that reads and writes reach speeds that approach read/writes of RAM.

Lifehacker has a nice little article on SSDs.

I still had all of the media which came with the system. To my less technical readers, I say this: keep the discs that come with your system – it makes your life unbelievably easier. The Windows install went amazingly fast, as expected with the reads from the Windows 7 DVD being the only bottleneck. After the first boot into Windows 7 (at very low resolution), I installed the chipset driver as Dell recommends.

It was here that the process started getting painful.

I started trying to work through the Dell Drivers and Utilities only to run into an error message indicating that the driver was not valid for my system configuration. The only drivers I could get installed were the network drivers. Note to Dell: I know what I’m doing. Windows is smart enough as well such that if I say install something, don’t tell me I can’t; you will only make me mad. At the urging of my wife, I put the computer down for the night and went to bed.

That night, something magical happened.

Those of you who remember previous versions of Windows and the mediocre functionality of Windows update know the pain of trying to play the game of connect the dots between drivers and hardware. The update often didn’t work, didn’t install correctly, or the driver just didn’t exist. However, the next morning, I woke up and picked up my system and discovered that all of the driver configuration and installation was complete.

Go me! Windows Update is NOW what it should be!

With that in mind, I ran through a checklist of details one must do for SSDs. Luckily, the drive already had the latest firmware, so I didn’t need to muck with that. The above Lifehacker article deals with the main things I needed to do.

Next, I decided to move the C:\Users directory to my D drive to save space. The best description of what to do can ALSO be found on Lifehacker (but before you follow that, keep reading)

I should note that this introduced me to the Windows command mklink. WHAT THE HECK!?!?! I’M ONLY NOW FINDING OUT ABOUT THIS? If you are a Unix or Mac user, it’s the same as the UBER useful ln command.

After doing this, and being rather pleased with myself, I rebooted. Aaaaaand boot failure. I couldn’t log in, my system was totally dead in the water. Furthermore, Windows repair couldn’t fix the problem.

*weeps softly*

Reformat, reinstall. I probably could have found a more elegant way to fix the issue, but I realized this would be faster. This time, after doing the “Move the users Directory”, I did one more trick – I did the Registry edit described here.

BINGO! I moved all of my data back on to my D: drive and enjoyed Windows booting in 9 seconds and various pieces of software install incredibly fast.

Next up, Backup software!

November 11, 2012
by Russell Sinclair
Comments Off

Your first automated web test!

This is the second post in a series started here.

Now that all the software from this post has been installed it is time to WRITE YOUR FIRST PIECE OF AUTOMATION!

I’m going to work under the assumption that you have installed the IE Dev Toolbar and that we are testing against Internet Explorer. By the way, you need to be connected to the internet to run this test.

Step 1:
Fire up Visual Studio and create a Console project called “SeleniumHelloWorld”.

Step 2:
Right Click on References and add references to each of the DLLs you downloaded the previous post: the Selenium Client Drivers for C#.

Step 3:
Open the Program.cs file and add the following code after the using clauses in your demo project

 using OpenQA.Selenium;
 using OpenQA.Selenium.IE;
 using OpenQA.Selenium.Firefox;
 using OpenQA.Selenium.Support.UI;

This will provide the library support for the Selenium Webtest.

Step 4:
In the Main method enter the following code

 IWebDriver driver = new InternetExplorerDriver();
 driver.Manage().Timeouts().SetPageLoadTimeout(TimeSpan.FromSeconds(30));
 driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30));
 
 driver.Navigate().GoToUrl("http://www.google.com/");

The first line is the declaration of the interface that will be running your test.
The next two lines are setting timeouts. I find this is helpful when running tests since sometimes Selenium will “jump the gun” when trying to find controls on the page. The first wait, “SetPageLoadTimeout”, changes the wait on the initial navigation. The second wait, “ImplicitlyWait”, sets how long after the page has loaded before the project gives up if it cannot find the control. The last line, starts IE and navigates to google.com.

Step 5:
This step represents 99% of your time spent doing web test automation. Start up Internet Explorer and navigate to google.com. You need to determine the code necessary to enter text into the search text box. Press F12 to start the IE Toolbar. On the Toolbar menu, select “Find” and click “Select Element By Click”. Then move your mouse over the text box and click. At this point the toolbar displays the line in the HTML of the code that represents the TextBox control. You are looking for the tag that looks like

 id="gbqfq"

This serves as the unique identifier we need to provide Selenium a way to find the control. The “gbqfq” is merely a variable name for our purposes. Each control should have an id tag. Not all do, so you may need to find another way to identify the control. Google has been good enough to provide us with one, so we may use it.

Step 6:
Type the following line:

 driver.FindElement(By.Id("gbqfq")).SendKeys("Hello World\n");

The “FindElement” is the workhorse that gets a pointer to the control on the page we want. It takes an argument that tells how to find the control. In our case, we are passing the static object/method “By.Id” and giving the the variablename we found in step 5. The SendKeys is command is simply the “type here”. We provided a carriage return so the routine will behave as if we’d typed in the code and pressed Enter.

Step 7:
Finish up with the following code:

 Console.WriteLine("Press any key to continue...");
 Console.ReadKey();
 driver.Quit();
 return;

This simply displays an instruction to the console window that will be open and waits for keypress. The line driver.Quit() closes the browser.

Now, you should be able to compile and run this code. You will need to be certain that you’ve placed the IE driver server executable in your path so that it will run.

I posted a copy of the code on Github for you to look at here: https://github.com/russellasinclair/SeleniumHelloWorld

You now have the primary foundation to write almost any web automation. In the future, I will write posts on more difficult automation problems.

I’ll be happy to answer questions posted below as time permits!

October 22, 2012
by Russell Sinclair
1 Comment

SQL: The Quick and Dirty Answer to Clustered vs. Non-Clustered Indexes!

While it may not be as necessary to know the real difference between the concepts of Clustered and Non-Clustered indexes in databases as you move away from the Database Administrator  role toward development and QA, it is a useful concept to understand. For the more technical minded, please forgive me, I’m going to take some liberties in order to get the concept across. It might hurt

In reality, the difference is fairly easy to define.

In a Non-Clustered index, you are essentially creating a look-up reference for a series of records based on whichever field (piece of data) you index. Think of an index as an address book: You want to find out where your friend “Joe” lives, so you look up “Joe” in your address book. Now you know where he lives and you can go RIGHT to his house and not have to knock on every door in your town until you find him. 

Great so what is a Clustered index?

To further extend our analogy, not only do you have an address book to find Joe, but everyone in the neighborhood has been sorted alphabetically. In short, Joe’s house is right next to John’s house, which is right next to Jose’s house. This means that if you needed to visit all your friends who’s name begins with “J”, you can do it more quickly because they all live in the same neighborhood.

In the non-clustered index, John and Joe could live in opposite ends of town. In a clustered index, they are “clustered together” in the same neighborhood.

Why would you choose one over the other?

Clustered:
Well, in a database, you can only have 1 field be clustered (actually you can, but that makes this harder to understand, so I’m glossing over it). Using our previous analogy of names, say our friends are named like this…

Joe Bob
John Zimmer
Jose Smith

You can tell them that all the people with first names beginning with “J” need to live in the same neighborhood. But you can’t also say that “Mary Barber” lives near “Joe Bob” because her first name begins with M. She lives in the “M” neighborhood.


Non-Clustered:
Lets face it, using our analogy, really, could also have a second address book that has everyone sorted by lastname first. This means, it’s easy for you to get “Mary Barber’s” address and then “Joe Bob’s” address, but it’s longer to visit them both because they are in different parts of town. The Non-Clustered Index, is just the address book.

Hopefully, this clears some things up for non-technical readers who hear their Database Administrator say “Yeah, I’ll just put a non-clustered index on the last name”. Then you can say, “Wouldn’t it make more sense to have last name be clustered?”

Are you the Database Administrator reading this? Let me know how miserable I’ve made your life by commenting below!

October 18, 2012
by Russell Sinclair
Comments Off

Automated Software Testing and YOU!

I listen to a lot of podcasts. Recently, “Yet Another Podcast“, Episode 70, Jim Holmes pointed out, that there is no good book on getting started with test automation.

That podcast played while I planned and constructed this site and I realized that:

  1. I enjoy writing
  2. I enjoy coding
  3. I know how to write test automation

With this in mind, I thought I should start sharing what I know about writing up test automation and how to get started with it. The world of software test automation is immense and overwhelming to someone just getting started with it. There are hundreds of tools out there of varying types and quality and usefulness. I want to demonstrate usage of some of the best tools out there with an emphasis on free/open source tools when possible due the fact that not only are they easily obtained, but also customizable as you, dear reader, have access to the source code.

One of the things I enjoy about writing software automation is that while it doesn’t have any truly complex code in practice (what Holmes refers to as “heavy lifting code”), it usually places a strong emphasis on pulling apart the software you are testing and trying to figure out how it works. By the same token, our code is going to be revisited for maintenance approximately twice as often as the software it is testing:

  1. We need to change our code when the software we are testing changes
  2. Our own enhancements and bug fixes

By the same token, we tend to be our own customers so we can tailor our code to fit our own “craziness”.

Initially, I plan to start with web test automation using Selenium Web-Driver. It is one of the more popular open source tools, is quite powerful, and supports the most widely used browsers in the US, if not all of them: IE, Chrome, Firefox, Safari, and Opera. Furthermore, it runs on Windows, OSX, Linux, and Solaris. Finally, it works with C# (my preferred language), Java, Ruby, and Python. I should also note that because of the way it works with C#, it’s conceivably possible to make it work with almost any .Net language.

My primary focus is going to be using C# in a Microsoft ASP.NET or MVC project. As I start, it will be helpful if you have at least a basic understanding of coding. You should be able to understand these posts if you have read any introduction to programming book (Bruce Eckel’s “Thinking in Java” or “Thinking in C#” are both good resources).

Later on, I’ll recommend other sources, but I don’t want to overwhelm you at this point.

WITH ALL THAT OUT OF THE WAY – LETS GET STARTED!

The first thing you are going to need is the various tools you will use. That will be the focus of this first post.

First, you need to download at least ONE browser. I am assuming you are a Windows user, so you have IE (Download IE). However, you are going to want a second browser. When working with web-testing, you will need to test the pages on different browsers. In today’s world, it is unlikely that your product will be used by only one browser. I’m going to recommend Firefox (Download Firefox) or Chrome (Download Chrome). Firefox is the easiest to work with as Selenium has native support for it. If you are going to pick a third, I’d recommend Chrome mainly for it’s powerful javascript console (yes, get used to it, you are learning Javascript whether you like it or not).

First, head over to Microsoft and download and install Visual Studio Express for “Windows Desktop” (Download Visual Studio). This is the IDE where you are going to be spending most of your time for this first batch of exercises.

Next, head over to the main Selenium website and download

  1. The C# Selenium Client Driver
  2. The 32-bit Internet Explorer Driver 

These are collections of files that do not require installation, but you will be using these bits of code a great deal.

Finally, you are going to want to grab developer tool add-ons for your browser. You are going to rely on these heavily in identifying bits you are coding against.

Note that, for each of these, download/install them with the browser they are for. Its not technically necessary, but each of these install more easily that way.

  1. For IE, you need the Internet Explorer Developer Toolbar
  2. For Firefox you need Firebug
  3. For Chrome you need it’s toolbar

OK, now that you have installed all of that, it’s time to sit back, relax, drink your favorite frosty beverage and comment below on how I don’t know what I’m talking about!

P.S. and yes, I WILL talk about writing tests in future posts!

October 16, 2012
by Russell Sinclair
2 Comments

Another Coding Exercise

In the same discussion that I referenced in this post; I was asked the following: “Write a function, that, given three integers A, B and K, returns the number of integers within the range [A..B] that are divisible by K.” The idea being that if I had A=12; B=19, and K=6, it would produce the answer “2″ because 12 is evenly divisible by 6, and the only other number in that range that is evenly divisible by 6 is 18. I had 30 minutes to write my answer. Initially, my 12 year old memory of my Discrete Math class let me down and I couldn’t remember the shortcut to do this. I would have Googled the answer, because I knew there was a trick, but I was concerned about academic honesty. My first approach was an algorithm that looked like this:

public int count_div(int A, int B, int K)
{
int retval=0;
for(int i=A;i<=B;i++)
{
if(i%K==0)
{
retval++:
}
}

return retval;

}

Quick to write, easy to understand, brute force answer. However, when the range was very large (A=1, B=100,000,000) it timed out. OK, the answer boiled down to remembering my Discrete Mathematics class from 2000. With about 8 minutes to spare, I finally finished puzzling out the answer of “Well really, it would be dividing B by K, minus A by K”. So, my next pass was the following:

public int count_div(int A, int B, int K)
{
int a;
if(A==0)
{
a=1;
}
else
{
a=A;
}
return (B/K)-(a/K);
}

 

I was nervous because this was an interview, I was running short of time, and made the trivial mistake of thinking “If A is zero, this won’t produce the right answer”. Of course, as you can guess it still didn’t produce the right answer. However, I’d solved the speed issue. I entered a few test cases and changed it to the following.

public int count_div(int A, int B, int K)
{
int a;
if(A==0)
{
a=1;
}
else
{
a=A;
}
return (B/K)-(a/K)+1;
}

 

Again, still wrong, but I didn’t have time to throw test cases at it to determine the full extent my wrongness. For some reason, the interviewer accused me of creating 2 variables, when I’d only created a single variable. I think he’d pretty much written me off and didn’t look at the answer nor care how close I was to the correct answer. While this didn’t produce the correct answer, I was very close. The problem I was running into boiled down to (12, 19, 6) and (13, 20, 6) do not give the same answer, despite the same range. That “+1″ ended up messing me up even more than the a=1 from a purely logical perspective. In hindsight, I sat back, and checked my work and kicked myself at how close my answer was.

public int count_div(int A, int B, int K)
{
return (B/K) – (A/K) + (A%K == 0 ? 1: 0);
}

 

Does this work? We can look to Software Quality Assurance knowledge and do a little equivalence partitioning to test it with a minimal number of cases. Equivalence Paritioning is when you divide the input data into “partitions” of data to create test cases. While this makes some assumptions, it can drastically reduce the number of test cases being developed. In the cartesian product of possible cases, the below cases handles the four basic situations that should appear. We know that the C# math functions generally work, so we shouldn’t need to test those.

Test Case 1: A = 6, B=11, K = 3
This case represents when A is counted in the answer, and B is not.
This case produces “2″ because

(11/3) – (6/3) + (6%3 == 0 ? 1 : 0) = 3 – 2 + 1 = 2.

Test Case 2: A = 7, B = 12, K= 3
This case represents when B is counted in the answer, and A is not
This case produces “2″ because

(12/3) – (7/3) + (7%3 == 0 ? 1 : 0) = 4 – 2 + 0 = 2.

Test Case 3: A = 7, B = 11, K = 3
This case represents when neither A nor B are counted in the answer
This case produces “1″ because

(11/3) – (7/3) + (7%3 == 0 ? 1 : 0) = 3 – 2 + 0 = 1.

Test Case 4: A=6, B=12, K = 3
This case represents when both A and B are counted in the answer
This case produces “3″ because

(12/3) – (6/3) + (6%3 == 0 ? 1 : 0) = 4 – 2 + 1 = 3.

Now, after about an hour of work (counting the original interview time but not counting writing this post) I have come up with the correct answer and created 4 possible unit tests to verify the code works.

Thoughts? Comment below!

October 10, 2012
by Russell Sinclair
Comments Off

SQL and “where not in” alternative

This question comes up occasionally, and I usually end up being a tad evasive about it. So, it comes time to post the real answer here:

“Lets say you have a student table and teacher table. How do you write a SQL query to show all teachers that don’t have a student?”

I always say, “Well the first thing that comes to mind is

SELECT t.teacher_id FROM teachers t
WHERE t.teacher_id NOT IN (SELECT s.teacher_id FROM students s)

but I know there is a more performant method.” Then I don’t follow up with the answer.

Yeah, I know…

The above query basically works, but it is slow because it has to pull up a list of all student records, extract the teacher_id, and then take the list and apply it to the teacher table. It is a very slow approach despite being VERY clear to the maintenance programmer. 

So, lets write up the more performant answer now!

SELECT t.teacher_id FROM teachers t
LEFT JOIN students s ON s.teacher_id = t.teacher_id
WHERE s.teacher_id IS NULL

Now the query says “OK, lets do a join and just throw out the places where there is no corresponding teacher_id” It is MUCH, MUCH quicker because it runs as a “single query”, instead of basically breaking down into 2 queries.

However, I would argue that if you were writing SQL for testing purposes, and not production purposes, you would want to use the first of the two examples.

Yeah, I know….

There is a real reason for it. The average QA person, even one who specializes in automation, has very different goals than the average developer. A developer wants to write the fastest SQL query possible because the end user wants their answer and the query may be run hundreds of times a second. On the other hand, a QA Engineer wants the answer, and has a bit more luxury in their time. Furthermore, if the QA is writing code for automation, then their primary goal is correct functionality and maintainability. The QA may need to use the faster solution, but the greater concern is clarity.

Got a better answer than me? Leave a comment to flame me below! I welcome the knowledge!