Improving Windows Cmd Experience

Having worked in Unix world and then moved to windows, I have always felt
the pain of not having a good shell. Windows command is painfully bad.
Even copying and pasting is difficult.

The things that miss the most from the unix shell are:

    - command history - the command history in default windows cmd is very
      restrictive. The history is not maintained across restarts of the
      cmd. Also, within on season, one has to use both up and down key.
      Navigating back in history will cause the position in the history to
      change, rather than add the command back to the history queue.

    - reverse search- using up and down key is very combursome
      especially for long history. A reverse search helps in saving
      time.

    - Powerful auto completion - hitting tab in windows cmd does weird
      things. It cycles through the list instead of completing the word
      till the point where the differences start.
   
    - Tabbed and Split screen- I use multiple cmd windows. Not having
      tabbed/ split view is a very bad loss of productivity, since I have
      to keep switching windows. Using alt-tab to switch between windows
      cycles through a large list of windows that I have kept open and
      this consumes time. Using a mouse is a no-no.

 etc among tons of other features.

Windows PowerShell solves the command history part of the problem, but
it doesn't quite feel like unix shell.

Recently, a coworker suggested to me to have a look at conemu, a console
emulator. This allow one to use split windows and tabs. It supports copy
pasting from the cmd window. You can also customize the color scheme.

Galvanised by this, I started searching for a similar tool for reverse
search. I found clink.

All one has to do is install clink, and it automatically adds itself to cmd by
injecting a DLL into it. The DLL then hooks the WriteConsole() and
ReadConsole() Windows functions.

With the combination of ConEmu and clink, I now have a good shell
replacement.

Thanks a ton for such tools. Also, I am sucker for tools which improve
productivity !!!

Visual Studio, ReSharper And Cache problems

Sometimes when using ReSharper and Visual studio, you will notice that there
is a lot of unresolved code or absence of ReSharper inspections in some files.
This is because of corrupted cache files of ReSharper. ReSharper caches a lot of
its data like results of Code Analysis, so that it can quickly load things and
need not rescan everything again. It stores the cache in the solution folder or
in a temp folder which is usually the local app data folder.

In such a case of unresolved code and inspection errors int ReSharper, one can
use the ReSharper menu to clear the cache.

                         ReSharper | Options | Environment | General

After clearing the cache you need to
reopen your solution and wait a little while ReSharper rebuilds the cache.
Usually this will solve the problem. But sometimes even this doesn't help.

In such a situation, only option is to locate the ReSharper caches directory and
remove the content and restart visual studio. The location of the cache is in
the app data folder :

              %LOCALAPPDATA%\JetBrains\ReSharper\v8.2\SolutionCaches\

 
Only thing that will differ would be the version number of ReSharper in the
above path. In the past few days, I have been doing this a lot and it was time
consuming enough to bother me. So I wrote a bat file to do this for me...

rem @echo off
set tasklist=%windir%\System32\tasklist.exe
set taskkill=%windir%\System32\taskkill.exe
set procName=devenv.exe

:STOPPROC
    set wasStopped=0
    set procFound=0
    set notFound_result=ERROR:
    
    for /f "usebackq" %%A in (`%taskkill% /IM %procName%`) do (
      if NOT %%A==%notFound_result% (set procFound=1)
    )
    if %procFound%==0 (
      echo The process was not running.
      goto :DELETE
    )
    set wasStopped=1
    set ignore_result=INFO:
:CHECKDEAD
    "%windir%\system32\timeout.exe" 3 /NOBREAK
    for /f "usebackq" %%A in (`%tasklist% /nh /fi "imagename eq %procName%"`) do (
      if not %%A==%ignore_result% (goto :CHECKDEAD)
    )
    goto :DELETE

:DELETE
set folder="%LOCALAPPDATA%\JetBrains\ReSharper\v8.2\SolutionCaches"
cd /d %folder%
for /F "delims=" %%i in ('dir /b') do (rmdir "%%i" /s/q || del "%%i" /s/q)

START "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe" "<path to your sln file>"

pause
goto :EOF 

call :STOPPROC

Now whenever I find such ReSharper problems, all I have to do is to run
this bat file. This even works when there are multiple instances of visual
studio running, it kills all the instances.

Do let me know if you have a better solution ....

Log Files

In today's world, the amount of log data generated by any application is
so much that it is not possible to open the said log files in a text
editor of choice. The modern logging framework like log4net make it
possible for the log files to be split based on size, so that, one can atleast
open them in a text editor. But the problem with this is, there is most often,
a case of explosion of number of log files and pretty soon we will have GBs
worth of log files all over the place. The whole logging issue becomes a mess
once we start to have a distributed system. Looking through multiple log
files from different sources and making sense of them is a pain.

Recently, I faced exactly the same situation. I found myself sifting
through GBs of log data generated by a system which had been running for
many days.

In a multi threaded environment, logs are written to a log file by
multiple threads and sequence of operations from a single thread will not
be available in consecutive lines. If the logging is extensive, then the
sequence of operations that we are interested in, might also be a subset
of the entire log file. Sifting through the log file for this is very time
consuming if done manually. Especially, if the text editor used is a very
basic one, with no advanced features to do marking, highlighting and
searching.

I use VIM, so I get by very well, for sifting through huge log files.

Nevertheless, I found myself doing this activity so often, that I had to think
of making a small tool for this.

The basic set of needs that I had were :
- Should be able to look into multiple files
- should be able to look into large files
- should be able to filter out only those lines that I am interested in based
  on REGEX pattern matching.
- should produce a consolidated result file.

So I came up with this small utility class to do this for me :

Code 

1. public class LineFilter : ILineFilter
2. {
3.     private readonly string[] fileNames;
4.     private readonly List<string> searchFilter = new List<string>();
5.  
6.     private LineFilter(params string[] fileNames)
7.     {
8.         this.fileNames = fileNames;
9.     }
10.  
11.     public ILineFilter WithFilter(string filter)
12.     {
13.         searchFilter.Add(filter);
14.         return this;
15.     }
16.  
17.     public void WriteResults(string outFileName)
18.     {
19.         var regex = string.Join("|", searchFilter);
20.         var g = new Regex(regex, RegexOptions.IgnoreCase);
21.         var linesToWrite = new List<string>();
22.         foreach (var f in fileNames)
23.         {
24.             Console.WriteLine(f);
25.             var lines = File.ReadAllLines(f);
26.             foreach (var line in lines)
27.             {
28.                 var m = g.Match(line);
29.                 if (m.Success)
30.                 {
31.                     linesToWrite.Add(line);
32.                 }
33.             }
34.         }
35.         File.WriteAllLines(outFileName, linesToWrite);
36.     }
37.  
38.     public static ILineFilter CreateLineFilter(params string[] fileNames)
39.     {
40.         return new LineFilter(fileNames);
41.     }
42. }

The way to use this class is as follows :

Code

1. LineFilter.CreateLineFilter(
2.             @"d:\MyLogFile-Debug-2015-04-29.11.txt",
3.             @"d:\MyLogFile-Debug-2015-04-29.10.txt",
4.             @"d:\MyLogFile-Debug-2015-04-29.9.txt",
5.             @"d:\MyLogFile-Debug-2015-04-29.8.txt",
6.             @"d:\MyLogFile-Debug-2015-04-29.7.txt",
7.             @"d:\MyLogFile-Debug-2015-04-29.6.txt",
8.             @"d:\MyLogFile-Debug-2015-04-29.5.txt",
9.             @"d:\MyLogFile-Debug-2015-04-29.4.txt",
10.             @"d:\MyLogFile-Debug-2015-04-29.3.txt",
11.             @"d:\MyLogFile-Debug-2015-04-29.2.txt",
12.             @"d:\MyLogFile-Debug-2015-04-29.1.txt",
13.             @"d:\MyLogFile-Debug-2015-04-29.txt")
14.     .WithFilter(@"starting task A [A-Z0-9]{8}-([A-Z0-9]{4}-){3}[A-Z0-9]{12}")
15.     .WithFilter(@"total DB connections are")
16.     .WithFilter(@"task A [A-Z0-9]{8}-([A-Z0-9]{4}-){3}[A-Z0-9]{12} actions")
17.     .WithFilter(@"task A [A-Z0-9]{8}-([A-Z0-9]{4}-){3}[A-Z0-9]{12} ends")
18.     .WriteResults(FileName);

As you can see above, it is very convenient to use this. No matter how
many files I have, it will filter out the unwanted lines and generate
a single file from it. Of course, if the filter is letting through a lot
of lines then the number of lines in the generated file is going to be
large. This means the generated file will also be large. We can split the
writing to the output file and make multiple files and this is pretty easy
to do. In the above example I have used the REGEX for filtering the GUID.

I am a big fan of REGEX, particularly for the searching of log files.
A big use of this log filtering utility is that I can take output of the
tool and feed it to a graphing software to generate graphs. All logs have
a timestamp as their first column. So if we can filter logs indicating
some type of start and end of actions, then the generated log file will
look like:

Log

1. 2015-04-29 13:54:12,123 [Thread 1] Starting Task A 9e8fa525-5eeb-4653-b9f6-a489017eada8:
2. 2015-04-29 13:54:42,123 [Thread 1] Task A 9e8fa525-5eeb-4653-b9f6-a489017eada8 ends

This can used by the graphing software to generate graphs of when
"task A" started and ended and how long "task A" took.

This class is not perfect, but it does the job for me. I see many improvements
that can be done to this utility class :
- Splitting of the output file into many files if the size of the output
  file is large.
- Speeding up the whole process by using multiple
  threads.
- Reordering the lines of the generated log file based on
  further pattern matching.

I have ideas with respect to 3rd point above, I shall write about it soon..

I only wrote this small utility to scratch my own itch. Of course
I recommend a better logging strategy,like logstash.net for highly
distributed systems . The utility here is only for making offline processing of
log files and generate meaning from them. For example, go through 4 days
of worker logs and make a graph of the timings of actions.

Practice makes a man perfect - CodeKata

A kata is an exercise in karate where you repeat one particular form, multiple
times, each time making little improvements. The intent behind code kata is
similar.

How did Sachin Tendulkar or Roger Federer become the greatest players in their
sport?

Through practice. They practiced everyday. They did, even after they became
no.1 in their sport. It is an accepted norm to practice in the sports industry.
However, this doesn't happen in the software industry. A software
engineer doesn't practice his profession daily. Everyone thinks  "I write software
daily, so why practice writing more software ??"

Sadly, this is not wise to say. One has to practice .. to get better... to learn ..
to stay relevant .. to stay current

And so begins my journey on Code Kata. Inspired by codekata.com, which has many
interesting problems to practice on. Today, I am attempting to solve problem 1.

Kata01: Supermarket Pricing

Using some of the concepts of DDD, I tried to come up with a ubiquitous language
that might fit the model needed here based on the information presented.

"A super market order contains many items, which depict the products, their
quantity and their billing amount based on a pricing scheme"

This leads to the following model:

    - An order has many items
    - An item has a product, its quantity
    - A product has the pricing scheme used to calculate the amount.
    - A pricing scheme has many types
        - Single e.g. this can of beans costs $0.65
        - Multiple e.g three for a dollar
        - NonCountable e.g $1.99/Pound
        - MultipleWithOffer e.g buy two, get one free

This is shown in the UML below :

The clear advantage is that calculation of the total bill amount and individual
line item's amount easily fits in a strategy design pattern. Adding more
pricing schemes becomes easy, Software becomes more composable.

Let us now answer the questions posed in the code kata 1 :
   
    - In the case of Multiple pricing scheme, like "three for a dollar", what is
      the price if I buy 4, or 5 pieces?
            A: It is the responsibility of the algorithm in the "Multiple"
            pricing scheme to decide this based on the business rules. for
            e.g. 4th and 5th one will have price of what a single piece would
            have if only one piece was bought.
    - In the case of  NonCountable scheme, like $1.99/pound, what does 4
      ounces cost?
            A: Again calculated by the "NonCountable" algorithm, which would
            have conversion methods to different fractions of the measure and
            also to different units of the measurement. Adding a conversion
            algorithm for a newer unit later would just mean adding a method
            and/or decorator pattern.
    - For MultipleWithOffer, like "buy two, get one free", does the third item
      have a price?
            A: maybe, depends on the business rules.

This was my kata for today, trying to practice my DDD and modelling skills.
I don't know if this approach mentioned is the best way to model the
supermarket pricing. However, I know  practicing daily will surely make me
better at modelling software.

Happy Practicing !!!

To-do and task managers - plenty of fish in the sea

There is no dearth of to-do and task manager applications in this world. All boast a lot of features, and most of them get it wrong, Fancy UI is not the key point in such apps, but the workflow that it provides is.

If one has a look at the typical workflows of people with respect to tasks and to-dos, then it becomes abundantly clear, what are the features that these apps must be concentrating on.

Primary reason why people use to-do managers is that human brain is not designed to remember things and do it before the task becomes overdue. For e.g. if we need to buy new batteries for the torch, human mind will only remind us that we need new batteries, when the power goes off and not when we are near the shop. For this reason, people use to-do managers so that they can look at the shopping list when they are near shop to check if the need any purchases to be made. Thus, To-dos must be accessible everywhere, on all my devices.

Let us look into the anatomy of a "task" There are many types of task that one usually performs.

• Something that one has to do on a given day. e.g. implement feature "x"
• Something that one does many times periodically. e.g. pay bill on 1st of month.
• Something to respond to. e.g. email, phone call, article on a browser.. etc
• Hierarchical to-dos, like sub projects under projects etc,
• Merging of calendar and to-do view to get a better view of the day

In my opinion, Point 2 is extremely important, If a to-do manager does not have recurring tasks, then it is not a good to-do manager. By not having recurring tasks with good customizable scheduling rules, it is only subjecting the user to a lot of manual scheduling and wasting the users time. Imagine a task/time management software wasting time !!!

This means, a good to-do manager, is one, which runs/syncs on multiple platforms, with features such as recurring tasks. It should have ability of importing tasks from various sources like email, phone, browser etc. It should also support hierarchical tasks and single view of calendar and tasks.

I recently discovered this great app called "todoist", which satisfies all the above conditions. It is a very nice app and developed by understanding the problems of the users instead of just a fancy UI.

Kudos to the todoist guys !!

Building Software With Ease

The problem with software for me is that we dont have anything like Lego blocks
where we can just pick some blocks up and make some bigger block. Before you
dismiss my theory, let me explain what I mean by pick some block and make
something. In Lego, we get standard blocks and those blocks have standard ways
it can connect to other blocks. Using these we can connect one block to another
and more. I dont care what meterial, colour, texture the blocks are made of ? As
long as they have the same standard way to connecting to each other, I can build
anything.

What I am looking for is a software world, where interfaces are standard defined
and we dont care which platform the individual software blocks are written in.
But when put together, they should be able to achieve the purpose...


The closest that we get in software to this is, I think, In Unix with pipes
where we can chain together multiple pieces of software.

I really want to see a world, where I get standard lego block sized software and
I can make a bigger piece of software by joining them together without any
effort.

Are microservices one of the solutions to this problem ?

Serializable isolation level

Continuing with the previous posts(1,2,3,4), Although repeatable read isolation
level can guarantee that dirty reads don't happen and that data read multiple
times in a transaction doesn't change within the transaction, repeatable read
isolation level still allows for a relation to change via something called as
"Phantom Tuples".

Phantom reads
A phantom read is said to have occurred when, during a transaction, two
identical queries are executed, and the number of rows returned by the second
query is different from the first. The phantom reads is a special case of
Non-repeatable reads when Transaction t1 repeats a range SELECT query and,
between both operations, Transaction t2 creates, by INSERT command, new rows
(in the table) which fulfill that SELECT clause.

Consider the items table again :

            items
    + -- + --------- + ----- +
    | id | item      | price |
    + -- + --------- + ----- +
    | 1  | Colgate   | 5.23  |
    + -- + --------- + ----- +
    | 2  | Pepsodent | 4.33  |
    + -- + --------- + ----- +
    | 3  | Sensodyne | 12.33 |
    + -- + --------- + ----- +

Consider the following transactions:

 ------------------------- + ------------------------- +
| transaction t1 statements | transaction t2 statements |
+ ------------------------- + ------------------------- +
| SELECT * FROM items       |                           |
| WHERE price < 12.33       |                           |
| //selects Colgate         |                           |
| //and pepsodent           |                           |
+ ------------------------- + ------------------------- +
|                           | INSERT INTO items VALUES  |
|                           | (4, Close-Up, 3.33        |
|                           |                           |
|                           | Commit;                   |
+ ------------------------- + ------------------------- +
| SELECT * FROM items       |                           |
| WHERE price < 12.33       |                           |
| //now 3 rows are found    |                           |
| Commit;                   |                           |
+ ------------------------- + ------------------------- +

Note that Transaction t1 executed the same query twice. But still number of rows
returned during the two select queries is different. This might seem wrong, but
it is justified as per the definition of repeatable read isolation level, as the
same values are read both the times, but only that extra rows are also read.
The records that originated in the second select query which were not present in
the result of first statement of transaction t1 are called phantom tuples. They
seem to have emerged from nowhere as far as transaction t1 is concerned(Hence
phantom).

Note however if we replace the statement in the transaction t2 with :

    DELETE FROM items WHERE price = 12.33;

The deletion will either happen before entire transaction t1 or after
transaction t1. This is because of how repeatable read is implemented using
locks. This means values read in first statement of t1 are locked. In this case
DELETE cannot go through in between the two statements of transaction t1.
However, in the case of INSERT, the new values being inserted are not locked. If
a range based lock is also held apart from read-write lock, then the situation
of inserting phantom tuples will not occur.


Locks vs isolation levels
In the case of a read uncommitted isolation level, no locks are held. In the
case of read committed, a write lock is held until the end of transaction, so
that no other transaction can write concurrently. In repeatable read, both read
and write locks are held till the end of transaction, so already read rows
cannot be changed. However, change in the range of rows is possible.

If both read and write locks are held and range change is also not allowed by
holding range locks, then the isolation level is called "Serializable".


Isolation Levels vs Read Phenomena
+ ------------ + ----------- + -------------- + --------- +
| Isolation    | Dirty reads | Non-repeatable | Phantoms  |
| level        |             | reads          |           |
+ ------------ + ----------- + -------------- + --------- +
| Read         | may occur   | may occur      | may occur |
| Uncommitted  |             |                |           |
+ ------------ + ----------- + -------------- + --------- +
| Read         | -           | may occur      | may occur |
| Committed    |             |                |           |
+ ------------ + ----------- + -------------- + --------- +
| Repeatable   | -           | -              | may occur |
| Read         |             |                |           |
+ ------------ + ----------- + -------------- + --------- +
| Serializable |             |                | -         |
+ ------------ + ----------- + -------------- + --------- +

Repeatable read Isolation Level

Continuing with my previous topics(1,2,3), I would now like to write about the
problem with the "read committed" isolation level. Read Committed guarantees
that a data that is read by a transaction is already committed by the time it is
read. However, it doesn't guarantee that, if the transaction performing the
read, issues a re-read, it will find the same data.

This is called a non-repeatable read, one of the 3 read phenomenons mentioned in
the SQL standard.

Non Repeatable Read
A Non Repeatable read is said to have occurred, if a transaction performs two
reads of the same data withing a transaction, and the values of the data differs
between the two reads. This means some other transaction has committed the data
in between the two reads.

Let us consider the following transactions, on the same table as used in the
previous post.

            items
    + -- + --------- + ----- +
    | id | item      | price |
    + -- + --------- + ----- +
    | 1  | Colgate   | 5.23  |
    + -- + --------- + ----- +
    | 2  | Pepsodent | 4.33  |
    + -- + --------- + ----- +
    | 3  | Sensodyne | 12.33 |
    + -- + --------- + ----- +

+ ------------------------- + ------------------------- +
| transaction t1 statements | transaction t2 statements |
+ ------------------------- + ------------------------- +
| SELECT * FROM items       |                           |
| WHERE price = 12.33       |                           |
| //selects Sensodyne       |                           |
+ ------------------------- + ------------------------- +
|                           | UPDATE items set price    |
|                           | = 5.66 WHERE item =       |
|                           | 'Sensodyne';              |
|                           | Commit;                   |
+ ------------------------- + ------------------------- +
| SELECT * FROM items       |                           |
| WHERE price = 12.33       |                           |
| //now row is found        |                           |
| Commit;                   |                           |
+ ------------------------- + ------------------------- +

If t1 makes a read before the commit of t2 and then makes another read after the
commit of t2. The second time t1 reads the value, the "Read Committed" isolation
guarantees that the value that is read is already committed one. It can easily
be noted that the second time, the read of t1 finds no item, whereas the same
read had one item in the first read.

We can ensure that this doesn't happen, if we use a higher isolation level
called "Repeatable Read". If a transaction is having isolation level Repeatable
Read, it means that the transaction cannot perform dirty reads, but also
multiple reads of values in the same transaction results in the same values. So,
the second read statement of t1 will also return the value Sensodyne, even
though the value was changed by a commit due to another transaction.

This isolation level is usually the default value in most database systems.

Note that even "Repeatable Read" Isolation level doesn't guarantee the global
serialization. I will write about this soon...

Read Uncommitted and Read Committed Isolation Level

In last post, I wrote about the different isolation levels of a transaction.

Now, I would like to write more about each one of them. But before we get into
that I want to define certain terms. You will find that the terms I want to
define are all relating to reading. This is because the isolation levels also
pertain to how a data is read by a given transaction. Hence, ANSI SQL standard
defines the following read phenomena:

    - Dirty Read
    - Non-repeatable reads
    - Phantom reads

Understanding how the read phenomena works, will lead us to understanding how
the different isolation level work.


Dirty Read
A data is deemed as "dirty" when it has been modified by a transaction and that
transaction has not yet been committed. When a transaction reads an item that is
made dirty by another transaction, a "dirty read" is said to have occurred.

Consider the table below :

                                        items
        + -- + --------- + ----- +
        | id | item      | price |
        + -- + --------- + ----- +
        | 1  | Colgate   | 5.23  |
        + -- + --------- + ----- +
        | 2  | Pepsodent | 4.33  |
        + -- + --------- + ----- +
        | 3  | Sensodyne | 12.33 |
        + -- + --------- + ----- +


In the above example, let us consider a situation in which transaction t1
changes a row, but does not commit these changes and Transaction t2 then reads
the uncommitted data. Now if Transaction t1 rolls back its changes, which has
already read by Transaction t2, or updates data again with different values,
then the view of the data may be wrong in the data structures of
Transaction t2. This is shown below :

 + ----------------------------- + ------------------------- +
 | transaction t1 statements     | transaction t2 statements |
 + ----------------------------- + ------------------------- +
 | UPDATE items SET price = 6.66 |                           |
 | WHERE item = 'Colgate';       |                           |
 + ----------------------------- + ------------------------- +
 |                               | SELECT price FROM items   |
 |                               | WHERE item = 'Colgate';   |
 |                               | // this reads 6.66        |
 + ----------------------------- + ------------------------- +
 | ROLLBACK;                     |                           |
 | // because of DB crash or     |                           |
 | // application error          |                           |
 + ----------------------------- + ------------------------- +

This brings us to the first Isolation level of "Read Uncommitted". When the
isolation level is "Read Uncommitted", a transaction can perform dirty reads.
One has to be extremely careful when setting the isolation level of
a transaction to this level, as the values read by the transaction may not be
accurate, since these values that are read may not actually exist, as the other
transaction may have rolled it back. The apparent advantage that we gain by this
isolation level is higher level of concurrency at the expense of data
consistency. This is the lowest/weakest isolation level, as in, one transaction
is not well isolated from the other.

In the example that I have mentioned above, transaction t2 has an isolation
level of "read uncommitted". It doesn't matter what the isolation level of t1
is. As already mentioned isolation level pertain to reading data.

To overcome this problem of reading inconsistent data, we can restrict the
transaction to only read the data that has been committed. This prevents reading
intermediate data. This is the next isolation level "Read Committed".
A transaction with isolation level set to "Read Committed" will not perform
dirty reads. "Read Committed" isolation is stronger than "Read Uncommitted" but
allows for less concurrency.

Using a "Read committed" isolation level still doesn't guarantee global
serialization. I will write about that soon...

Isolation levels and transactions

I wrote about transactions in databases a while back, along with the ACID
properties that govern them. The whole idea of writing that post was to give an
introduction to the post that I am about to write now.

I stumbled upon a legacy piece of code at work, well, it is not legacy but more
of a layer underlying the piece of code I write. Here, I noticed that the
transaction was started with a parameter passed to the method "BeginTransaction"
to indicate an isolation level to be used with the transaction. Unfortunately,
I did not know anything about "isolation levels". This led me to go on a journey
to learn about Isolation levels in a transaction.
Most of the stuff I write about are something that I learnt newly.


What is Isolation ?
A transaction in a database is a sequence of DB operations that is performed as
a single logical unit of work. A transaction satisfies all the ACID properties.
One of the ACID properties is "Isolation". A piece of data, which a transaction
must work on, either for reading or modification, must either be in the state it
was in before another concurrent transaction modified it, or after the second
transaction has completed, but it never will be in an intermediate state. This
is referred to as "Serializability". If 3 transactions run concurrently, and, they
produce the same outcome, as if they were run in some serial order, then it
means that they satisfy the serializability property.

Drawbacks of Serializablility
Although serializability gives us predicable and understandable behaviour along
with the consistency of the database, it also adds a considerable amount of
overhead. This is because of reduced concurrency. Serializablility is most
commonly achieved by using locking mechanisms.

Overcoming reduced concurrency
To overcome reduced concurrency, database systems allows the user to make
a choice to use a weaker level of isolation. This means that, it is under the
application programmer's control to increase concurrency if the overhead is very
high. Since Isolation is most commonly achieved by using locking mechanisms,
reducing overhead implies reducing the isolation levels, which means
a reduced degree of locking will be used. The application programmer must
carefully analyze database access patterns in the software to ensure that
any reduction of isolation level does not cause software bugs that are
difficult to find. On the other hand, if higher isolation levels are used,
the possibility of deadlock is increased, which also requires careful
analysis.

Isolation levels
Four isolation levels are usually provided by database systems as governed by
SQL standard.

        - Serializable
        - Repeatable read
        - Read committed
        - Read uncommitted

"Serializable" is the highest isolation level, where as "read uncommitted" is
the lowest. This means, as one traverses the above list from top to bottom, the
overhead reduces. If one traverses the list from bottom to top, then the
consistency increases. Stronger isolation levels are at the top of the list and
weaker isolation levels are at the bottom of the list. "Read committed" is
usually the default value of isolation level in database systems. As already
mentioned by me, I found that the isolation level can be set by passing the
appropriate parameter in the "BeginTransaction" call. This implies that
isolation levels are per transaction. Also, an isolation level set on one
transaction doesn't affect any other transaction. For e.g. if transaction t1
uses isolation level "Repeatable Read" and transaction t2 uses "Read
uncommitted", then the isolation level "Repeatable Read" set by transaction t1
is guaranteed to be applied on the transaction t1 and it will not affect the
behaviour of the other transaction t2 which will get its own isolation level
"Read uncommitted".

More about the different isolation levels soon....

Transactions in a Databases

A transaction in a database is a sequence of DB operations that is performed as
a single logical unit of work. Typically this means that the logical sequence of
operations must happen together or not at all. But there is more to this than
just that. What is the meaning of this logical unit of work ? For a long time
I thought transaction means making a bunch of actions an atomic operation, but
it is not.  This logical unit of work must exhibit four properties called the
atomicity, consistency, isolation, and durability (ACID) properties. Only when
a sequence of operations satisfy the "ACID test" ( or should I say litmus test,
pardon the pun) they can be collectively a "transaction".
 
Atomicity
A transaction must be an atomic unit of work; This means
if the operations A,B,C,D are performed in a transaction then all of A,B,C and
D are done, or none of them are performed. So, while performing operation
B, if there is a failure, due to a wrong query, for example, then whole
transaction is aborted and none of the result of actions A,B,C and D are
persisted. But what about the operation A which was already performed ? This
operation A is "rolled-back". However, If all of them are completed without any
errors, then the transaction is committed and the its effects are persisted.

Consistency
This brings us to the next item, which is consistency.
When completed, a transaction must leave all data in a consistent state. This
means that whatever modifications are performed in the operations A,B,C and D;
the data integrity of the relational database must be maintained. All the rules
that govern this consistency in the database must be passed. Moreover, All
internal data structures, such as B-tree indexes or doubly-linked lists, must be
correct at the end of the transaction.

Isolation
In today's world filled with massive software which are driven by high data
processing needs, concurrent access to databases and data is not uncommon.
In-fact it is foolish in today's world to not allow concurrent access to data.
Thus it not weird to have concurrent transactions running on the same database.
It is also common sense to understand that one transaction, if it has to have
the "Atomicity" and "Consistency" property, must be impervious to the
modifications made by another concurrent transaction. A piece of data, which
a transaction must work on, either for reading or modification, must either be
in the state it was in before another concurrent transaction modified it, or
after the second transaction has completed, but it never will be in an
intermediate state. This is referred to as serializability. What is
Serializability ? Consider a situation in which there are 3 transactions tA, tB
and tC that can run. Assuming that there may be dependencies among them, if the
3 transaction run concurrently and they produce the same outcome, as if they
were run in serial in the same order they completed when they were run
concurrently, then it means that they satisfy the serializability property.

Durability
After a transaction has completed, its effects are permanently stored in the
system. The modifications persist even in the event of a system failure. For
instance, if a transaction is already completed, then even if there is a system
crash, the transaction will remain in the state in which it was completed and
its state can be recovered from the database.