One of the most common question in moving towards Agile Development is "Where should I start from?". If you'll ask me, setup a Continuous Integration (aka "CC") would be the first thing you should start with.

Step 1 (Check for compilation bugs): Code Quality ~= 30%

The CC should be able to identify check-ins to your source control, get the latest source and compile it. The output should be either "green" (Everything compiles successfully with 0 warnings) or "red" (more than one warning or compilation errors). In addition to the fast feedback, the output should also include the files that were changed from the last build (and by whom, so people could know where to look).

The immediate value is priceless. the ability to SEE whether your source-code is stable enough to allow other programmers perform Get Latest and continue their work and the "fail-fast" attitude can save you a lot of time in the long run. It's important to realize though that even if the code compiles without warnings, it still doesn't mean that you could count on the quality of the code.

Step 2 (Check for component-based quality): Code Quality ~= 70%

If you can go one extra mile, write a few automated tests (via one of the available XUnit frameworks) for your components. This means that you are able to inject the component's dependencies from the outside and simulate mini use-cases on component's level. This step is crucial even if you write those after the code itself was written. Let the CC run them if Step 1 is OK. This should allow you to catch the majority of your bugs (I'll leave the "how to write good tests" to another post). If all is green, you know that the system behaves as expected, at least to some extent.

This step is not trivial as it requires you to design for testability and invest in proper testing. Don't let go of this step though, automated tests on this level will make your life much easier. It will take your code one (major) step ahead in "write code that could be changed". Agile is all about that state of mind.

Step 3 (Check for integration-based quality): Code Quality ~= 80%

Now that you're components are behaving as expected, you should try to write a few (automated, of course) tests that simulate the entire flow of 2 or more components (DB is a component as well) in the system. As the system grows and more uses cases are added, you should try to improve these tests as they give a solid proof that the SYSTEM works.

Step 4 (Make everything visible): Code Quality ~= 90%

The state & quality of your source control should be visible to the Team and Management as you want to insure IMMEDIATE response time in case someone check-ins a low quality code (on any level). Fixing a failing test three days after the change itself is a bad symptom of low visibility or low perception, by the Team, regarding the importance of the quality of the system.

Step 5 (Automated deployment): Code Quality ~= 95%

After successful build you would like to deploy the latest source on a dedicated environment which the developers could play with before deploying to another (testing?) environment. This won't be a stable environment, but at least it will give a quick look at the current state of the system - the way customers would see it.

Step 6 (Procedures checking): Code Quality > 95%:

You can add many more checks to the flow, such as Tests Coverage or FxCop. Leave those to the end. From my experience, Time .vs. Value in these features will vary from team to team. You'll gain much more from investing in Steps 2-3.

Semingo CC:

Each developer & manager on our team have a CCTray(the little red circle in the little picture above) which is either Red(source control is damaged), Yellow(Build in action) or Green(Life is sweet). We're using Cruise Control.Net, CCTray, MSBuild (and TFS plugin) and NUnit to perform all of the above. 

A few teasers:

* Image of Steve Urkel (from the famous Family Matters TV series) is shown for failing build.

* On the right you can find Pasha (with a V sign I've added), one of our finest hackers modeling a successful build. You can also notice the 582 green tests (including Integration tests) and 2 changes made by Sagie since the last build.

* Going to NUnit Details, you can get the full details:

I had to cut the pictures in order to keep a sane width for the post, but you can get the drift.

btw - Aviel, yet another Semingo hacker add a "Doh!" (Simpson) each time the CC is red on his computer. It can be quite funny (and scary, if fully concentrated on code).

A few pearls from the office:

• Pasha: Damn, I like TDD so much. Writing a 5 minutes piece of code takes about a day (sarcastic tone, of course).
  
• Aviel: in my case, code does not come from my head but rather from my heart.
  Pasha: I saw your code. It comes from your ass.
  
• Me: Integration environment is the Tamagochi of an Agile Team
  
• (Phone ringing, Pasha answers)
  Robert: I'm Robert.
  Pasha: I'm Robert.
  Robert: I'm Robert.
  Pasha: I'm not Robert, I'm Pasha.
  Robert: Oh, My bad.
  Pasha: Bye then.
  (end of conversation).

After almost 1.5 months of Scrum at Semingo (a baby startup), I decided to expose the way we work at the moment and talk about the adjustments we've made in order to suit Scrum to our needs.

I'll start with our implementation to the Daily Stand-up Meeting. No doubt, our meetings are pretty funny (most of the time) and create the right vibe for the Team. The one thing I love most about our meetings is that by the end of the meeting, I know what is the general work-plan for each member in my Team.

Now it's easy to know what I'm planning to complete today (I spend 5-10 minutes planning my day before the meeting), when & if I need to finish something earlier (or at least decide about interfaces after the meeting) in order to integrate with others, help out or ask for someone's else help(code review for example), stay after the meeting in order to talk about something that pop up during the meeting, remove impediments (if I can) and most importantly - have a good laugh before the day begins.

Daily Stand-up Meeting (aka DSM) structure at Semingo:

When: 
Every day at 10:30.
Where:
Meetings room.
Time Box:
15 minutes.
Attendants: 
Pigs only (Chickens can (only) listen)
On the table: 
Each Pig answer these 4(!) questions:
(1) What have I done since the last DSM ?
(2) What am I planning to do until the next DSM ?
(3) Impediments - what bothers me to work ?
(4) Am I on track?
     If someone feels that one of his tasks won't be finished as planned, a flag is raised so the Team could assist. 
     The same goes if someone feels that he's going to finish before the expected time. 
     This means that he could help out someone else or take a few extra tasks we did not plan for the current iteration.

Notes:
Only one Pig talks at a time and he leads the conversation if reasonable questions comes up. He (alone) has the power to stop a conversation if he feels the conversation stray from the DSM path. Team members can decide to talk about an issue that was raised during the DSM just after the meeting is finished.

What next (DSM planned improvements)?
(1) You late, you get (punished, that is): each team member that late to the DSM must wear the "I was late to DSM, I will serve you coffee today" sign on his shirt.
(2) Red-Back on track: If the conversation is getting out of control (too many jokes, drill-down conversations, more than 1 Pig talk etc) - the red button is clicked and each Team member is being electrified with 120V. Well no, but it could have been a nice feature right? Clicking the red button will make a nice GONG!! so we could move on. The Team agree to listen to the GONG and get back on track, so we could finish in time and keeping the DSM productive.

Gosh, I did not know Raymond Lewallen was reading my blog (I guess I should start writing some meaningful stuff and stop playing around ;)) but I'm more than happy to raise up to the challenge and talk about what I am doing in order to go to the next level.

In one of my post, What it takes to become a great developer, I mentioned the notion of "Be Eager To Learn". I don't consider myself as a good developer due to my natural skills (I don't think that I'm mediocre, but certainly not Larry Page). Starting 8 years ago as a little teenager at 15, I had to work my ass off in order to keep up and show the rest of the people I was working with that I'm just as good as they are. Reaching this goal, I wanted to show myself that I can be the best guy at the company.

Eight years passed and a lot have changed, but I'm still very much eager to get better and more versatile. One thing I'll always keep with me, as it proved it self so far, is the no-fear attitude and the (sometimes) ridiculous optimism. I'm not afraid of doing new things or changing positions when an "offer you can't refuse" knocks on my door. Life is short and you most grow each and every day. I'm still the same team player guy, although I can get over confident (aka arrogant) or raise my voice here and there. I care about my teammates and know when to say "I'm sorry". I work with my heart and hopefully my current and future teammates will forgive me for my faults.

I think that in the last few years I've learned a lot about myself, about the things that really intrigued me, that push me to excel. I love coding, I love talking with people, mentoring, lecturing about technologies or Agile methodologies, but most of all - I enjoy taking ownership of projects I participate in and making them successful. I'm looking to surround myself with people smarter than me, those that have natural gifts in them, and making them better.

Things I should do

I should try to get more organized in planning my time. I read a lot of books about self management but I don't feel like I'm practicing them as much as I should. I should really invest more time in myself, trying to set goals and constantly reviewing them. I'm leading the Agile a la Scrum at Semingo so I hope to use this work & review notion more in my life.

I should learn more about Agile, Scrum and XP. I've read a few great books about Agile\Scrum\Management but I still have a lot of unanswered questions. I know that these methodologies only offer some solutions but I don't believe we should enforce them. I believe in making our own Agile process at Semingo. That said, I do want to read more books from people with different experience, different ideas and best practices I could learn from.

I should definitely write more posts! (particularly about Agile\Scrum)

Things I want to do

TDD: getting better in it and start lecturing about it more.
Multi-threading: This one is a new set of skills I'm developing at my current job. Looking at the near future, this skill is crucial as a developer.
WCF: I need to use it in my current job and I have a lot of catch up to do.
Lecturing: At least 4-5 lectures a year looks like a solid goal at the moment.

Most of all, I want to make Semingo the best place to work at, to bring more amazing guys&gals to work with us and making an application that will change the way millions of people work.

Things I won't do

I think that it's getting clear to me that I do not want to be an external coach. I don't see myself coaching a team for a 2-3 months and then shifting to another team. I enjoy working with people and I take pride and strength in making things complete.

I won't stop talking and writing about software, practices and people as long as I have keyboard and working set of 1-N fingers available. Count on it!

Tagging these folks

Pasha Bitz, Shani Raba, Doron Yaacoby, Eran Nachum, Ken Egozi

One of the downsides of using lock is obviously performance. While locking an object, any other thread trying to acquire the lock on that object will wait in line. This can open up a deep hole to performance hit. Rule of thumb while working with locks is to acquire it as late as possible and release it as soon as possible. To demonstrate the order of magnitude bad usage of locks can affect your performance, I decided to write a little demo. 
So let's assume we have a component that is responsible for executing tasks while getting new ones in the process (on different threads). I tried to make this example as simple as possible. Let's start with our "task" class:

public class Task
{
   private int _id;
   private string _name;

   public Task(int id, string name) {
      _id = id;
      _name = name;
   }

   public int Id { // getter, setter }
   public string Name { // getter, setter }
}

We have a TasksRunner that's responsible for getting new tasks and saving it to internal list and executing the current tasks every X milliseconds (via timer). In order to simulate a real-life process, I've made sure that executing a single task is expensive. Let's start with the non-optimized solution:

public class TasksRunner
{
   private List<Task> _tasks;
   private System.Timers.Timer _handleTasksTimer;

   public TasksRunner()
   {
      _tasks = new List<Task>();

      _handleTasksTimer = new Timer(200); 
      _handleTasksTimer.Elapsed += new System.Timers.ElapsedEventHandler(_handleTasksTimer_Elapsed);
      _handleTasksTimer.Start();
   }

   public void AddTask(Task t)
   {
      lock (_tasks)
      {
         _tasks.Add(t);
         Console.WriteLine("Task added, id: " + t.Id + ", name: " + t.Name);
      }
   }

   //Execute the (delta) tasks in a thread from the ThreadPool
   private void _handleTasksTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
   {
      ExecuteCurrentTasks();
   }

   public void ExecuteCurrentTasks()
   {
      lock (_tasks)
      {
         foreach (Task t in _tasks)
            ExecuteSingleTask(t);
         
         _tasks.Clear();
      }
   }

   private void ExecuteSingleTask(Task t)
   {
      Console.WriteLine("Handling task, id: " + t.Id + ", name: " + t.Name);
      Thread.Sleep(1000); //simulate long run
   }
}

AddTask will acquire the lock on _tasks and add the new task to the list while ExecuteCurrentTasks will acquire the lock (on _tasks) and simulate real execution on the task. Notice that during the execution, calling AddTask will wait until the current execution will be finished. Using Roy's ThreadTester, we can run the following in order to notice the behavior so far:

static void Main(string[] args)
{
   TasksRunner runner = new TasksRunner();
   ThreadTester threadTester = new ThreadTester();
   threadTester.RunBehavior = ThreadRunBehavior.RunUntilAllThreadsFinish;

   Stopwatch watch = new Stopwatch();
   watch.Start();

   int numberOfTasksToCreate = 100;
   threadTester.AddThreadAction(delegate
      {
         for (int j = 0; j < numberOfTasksToCreate; j++) 
         {
            runner.AddTask(new Task(j, "job " + j));
            Thread.Sleep(100);
         }
      });

   threadTester.StartAllThreads(int.MaxValue); //wait, no matter how long

   Console.WriteLine("Total time so far (milliseconds): " + watch.ElapsedMilliseconds);
   Console.WriteLine("Tasks added so far: " + runner.TasksAdded);
   Console.WriteLine("Tasks executed so far: " + runner.TasksExecuted);
   Console.WriteLine("Waiting for tasks to end...");

   while (runner.TasksExecuted < numberOfTasksToCreate)
      Thread.Sleep(1000);

   runner.Shutdown();

   Console.WriteLine("done!");
   Console.WriteLine("Total time so far (milliseconds): " + watch.ElapsedMilliseconds);
   Console.WriteLine("Tasks added so far: " + runner.TasksAdded);
   Console.WriteLine("Tasks executed so far: " + runner.TasksExecuted);
}

Running this test will give us a very poor result for adding & executing 100 tasks takes around ~99 seconds.

No doubt, the lock on _tasks while executing each and every task in the list is too expensive as we're depend on ExecuteSingleTask (which is expensive by itself). This way, each new task we're trying to add must wait until the current execution is finished. An elegant solution to this problem, suggested by my teammate Tomer Gabel, is to use a temporal object to point to the current tasks thus freeing the lock much quicker. So here is an optimized version of ExecuteCurrentTasks:

public void ExecuteCurrentTasks()
{
   List<Task> copyOfTasks = null;
   lock (_tasks)
   {
      copyOfTasks = _tasks;
      _tasks = new List<Task>();
   }

   foreach (Task t in copyOfTasks)
      ExecuteSingleTask(t);
}

This little refactoring give us around ~11 seconds for adding & executing 100 tasks.

Smoking!

In my previous post, "How to mock static class or static member for testing", Eli replied that mocking static data can be easily achieved via TypeMock with no extra work around it. Although Eli raised a valid point while TypeMock is a very strong framework that you should check out, I still believe that designing for testability overcomes the basic need for testing your code. Trying to use some sort of powerful Profiler-based framework in order to mock your "wires" is a great thing, but I would use it only if refactoring the code will take too much time or effort that will beat the value of performing the change.

I would like to make my teammates think toward testability while designing the API.

Breaking your code so it will be easily tested will make things a little bit more complex in terms of dependency injection (object A will need to get IB in its constructor in order to work rather then creating new instance of B inside of it), but the allegedly weakness of breaking your code, making you stray from the pure OOP you've learned in "OOP for dummies" courses at the university, is the exact strength I see in TDD. Designing the code by writing use-cases, playing with the API, breaking classes for Single Responsibility and constant refactoring wins the purpose of testing solely or pure OO code. Don't get me wrong, pure OO is nice on the surface, but don't get fooled by it. You write code to create successful software, not successful practice of old paradigms.

TDD constantly demands to re-design the system and refactor it to fit the needs the system invocate, having the tests will back you up in the path. This way you can follow the guidelines of a sane development process:
1). Design the general architecture (the big picture)
2). Agree on interfaces between components
3). Unit-test user stories (use-cases) and drill down into the design by each test.
4). Build Integration tests to connect between components.

and so on...

Now, if you want to refactor existing code or add new behaviors, it's simple:
1). Refactor the code.
2). See that everything compiles.
3). Run unit & integrations tests.
4). If all is good - check-in your source.

Claiming that TDD-OO is not pure OO is fine by me. I would prefer TDD-OO based code on pure OO any day.

Pasha Bitz, friend, teammate, hacker and gifted(?) photographer took some photos on our last Daily Standup Meeting and a few more at our offices:

Daily Standup Meeting (I'm on the left, Sagie(aka Flash)  is on the right):

Our office door (notice: it's beta 0.1):

And finally, the new setup, although it's far from being ready & not updated (I've got 2 22' inch LCD at the moment):

More pictures we'll be posted soon...

You must admit, I'm a pretty cool (yellow) geek:

You can create your own avatar here.