Moti and I have decided to form an invite-only Scrum Clan.
We would like to tag Pasha Bitz (snapshot below) as the 3rd clan member.

Pasha, choose carefully, you can only tag one Scrum-Lover like yourself to this distinguish clan ;)

p.s - If you want to be part of the Scrum Clan, please drop a comment and you *might* get an invitation...

                                 Scrum Rules !

Let me start with an out-loud recap of this post: Agile is not something you can put on a bread nor is it "a certain path to success".

It's about STATE OF MIND.

If I had to describe the meaning of "Agile" to a new teammate I would say: Agile is a constant thinking about how we, AS A TEAM, can produce working features to our users with high quality within a short time-frame.

Don't worry:
It's really OK to provide only a subset of feature(s) in one sprint.
It's really OK to leave SOME designing\architecture issues for later on as long as the high-level architecture is good enough (=you're comfortable with it) to answer the big questions.
It's really OK to implement only two REALLY-DONE-HIGH-QUALITY features in one sprint over four semi-working-not-demoable features.

The key here though is not really the practices, it's about the big bullets(again, state of mind):

1). Produce value for your customers and adjust\adopt early.
2). Build a Team (self leadership).

These ideas are hard to implement and require special kind of people. Putting the Team in front of yourself is not a very job-secure attitude.  The ability to help your teammates, shift tasks, taking ownership, critisize yourself and your teammates and getting better, produce high quality design, tests and code - all of it - requires versatile people with unique state of mind (and unique abilities, of course). It's worth it. When things glue, it's a real magic; Things start to get going by the Team, improvements and features starting to come from the developers\QA\Graphic Designer, adjustments are made on a regular basis, changes are welcome and productivity is celebrated.

You can feel something is going right.
That's Agile.

The way WCF proxies are designed is to live until shi* happens.

Let's assume that we have a CalcualtorService with one method named Divide(int a, int b). Sasha, a cool programmer-dude, trying to produce some usefull software writes:

public MyCalcualtorForm : Form {

   private CalculatorProxy _calc = new CalculatorProxy();

   Calc_Click(...) {
      _calc.Divide(firstNumber, secondNumber);
   }
}

What is the first error you can think of that could happen? Yep, DivideByZeroException.
Once the proxy gets an exception, it enters into a "Faulted" state which makes the proxy unusable(=you cannot use it again).
The quickest solution is to work "by the book" and create a new instance each and every time we need to call the service:

Calc_Click(...) {
   using (CalculatorProxy calc = new CalculatorProxy())
      calc.Divide(firstNumber, secondNumber);
}

But what's bad in this solution?

1. Performance - you pay (not a lot but neither little) for each creation of the proxy. Sure, it will probably not be your bottleneck, but heck, why is it useful? Most of the time the proxy will not throw an exception and yet we need to create it every time just to avoid the faulted state scenario. 
2. Design - If we declare this exception BY CONTRACT, I would expect that the proxy will still be usable afterwards. Do we really want to return Enum\int\string as status instead of throwing exception just because of poor design?
3. TDD - you know that I'm in love with it. Now imagine Dependency Injection. Component A recieve ICalculatorProxy and use it to... calculate something. Working "by the book" is no good as we want to recieve an instance of the proxy from the outside in order to mock it. Right, so we inject a proxy from the outside (got to love Windsor) and life is pretty sweeet. Darn! Wait! one poor (even by design) exception and our proxy goes dead. Very un-TDDish of Microsoft.

I had to come with a solution as no one will take TDD away from me. I present to you ProtectedProxy: this little IL-code-at-the-end-of-the-day will able you to recover from faulted state by creating a new proxy on each exception thus making your proxy... useable (couldn't think about a better word to describe it). Think about a situation where your proxy is trying to call the service but the service is down; In Semingo, we decided that we want to keep trying until the service is up. Via ProtectedProxy, you can determine how many times do you want to recover and when you should finally kill the proxy. Oh yea, ProtectedProxy uses Windsor in order to create new proxies if needed and logging messages to log4net. Good stuff.

In the example above, all Sasha had to do was to:
1). Initialize the _calc field by:
        ProtectedProxy<ICalculatorProxy> _calc = new ProtectedProxy<ICalculatorProxy>(new CalculatorProxy());
2). call _calc via:
        _calc.Instance.Divide(firstNumber, secondNumber);

But enough said, code please:

// Written by Oren Ellenbogen (07.08.07) - trying to protect our proxies so they could recover from:
// (A) The service is not up yet, but we want to try again later.
// (B) The service throws (ANY) exception, we still want our proxy to function (bubble the exception, but still keep on working).
// Microsoft intended to use a NEW proxy per call, but for TDD this is not ideal as we would like to inject proxies from outside as mocks
// in order to simulate multiple scenarios.

#region using

using System;
using System.Reflection;
using System.ServiceModel;
using Castle.Core.Resource;
using Castle.Windsor;
using Castle.Windsor.Configuration.Interpreters;
using log4net;

#endregion

namespace Semingo.Services.Proxies.Helpers
{
   
   public interface IProxy : ICommunicationObject { 
      bool Ping();
   }

Hours of joy...

Almost forgot, on the next post - "How to TDD WCF code" - stay tuned...

One of the most common question in moving towards Agile Development is "Where should I start from?". If you'll ask me, setup a Continuous Integration (aka "CC") would be the first thing you should start with.

Step 1 (Check for compilation bugs): Code Quality ~= 30%

The CC should be able to identify check-ins to your source control, get the latest source and compile it. The output should be either "green" (Everything compiles successfully with 0 warnings) or "red" (more than one warning or compilation errors). In addition to the fast feedback, the output should also include the files that were changed from the last build (and by whom, so people could know where to look).

The immediate value is priceless. the ability to SEE whether your source-code is stable enough to allow other programmers perform Get Latest and continue their work and the "fail-fast" attitude can save you a lot of time in the long run. It's important to realize though that even if the code compiles without warnings, it still doesn't mean that you could count on the quality of the code.

Step 2 (Check for component-based quality): Code Quality ~= 70%

If you can go one extra mile, write a few automated tests (via one of the available XUnit frameworks) for your components. This means that you are able to inject the component's dependencies from the outside and simulate mini use-cases on component's level. This step is crucial even if you write those after the code itself was written. Let the CC run them if Step 1 is OK. This should allow you to catch the majority of your bugs (I'll leave the "how to write good tests" to another post). If all is green, you know that the system behaves as expected, at least to some extent.

This step is not trivial as it requires you to design for testability and invest in proper testing. Don't let go of this step though, automated tests on this level will make your life much easier. It will take your code one (major) step ahead in "write code that could be changed". Agile is all about that state of mind.

Step 3 (Check for integration-based quality): Code Quality ~= 80%

Now that you're components are behaving as expected, you should try to write a few (automated, of course) tests that simulate the entire flow of 2 or more components (DB is a component as well) in the system. As the system grows and more uses cases are added, you should try to improve these tests as they give a solid proof that the SYSTEM works.

Step 4 (Make everything visible): Code Quality ~= 90%

The state & quality of your source control should be visible to the Team and Management as you want to insure IMMEDIATE response time in case someone check-ins a low quality code (on any level). Fixing a failing test three days after the change itself is a bad symptom of low visibility or low perception, by the Team, regarding the importance of the quality of the system.

Step 5 (Automated deployment): Code Quality ~= 95%

After successful build you would like to deploy the latest source on a dedicated environment which the developers could play with before deploying to another (testing?) environment. This won't be a stable environment, but at least it will give a quick look at the current state of the system - the way customers would see it.

Step 6 (Procedures checking): Code Quality > 95%:

You can add many more checks to the flow, such as Tests Coverage or FxCop. Leave those to the end. From my experience, Time .vs. Value in these features will vary from team to team. You'll gain much more from investing in Steps 2-3.

Semingo CC:

Each developer & manager on our team have a CCTray(the little red circle in the little picture above) which is either Red(source control is damaged), Yellow(Build in action) or Green(Life is sweet). We're using Cruise Control.Net, CCTray, MSBuild (and TFS plugin) and NUnit to perform all of the above. 

A few teasers:

* Image of Steve Urkel (from the famous Family Matters TV series) is shown for failing build.

* On the right you can find Pasha (with a V sign I've added), one of our finest hackers modeling a successful build. You can also notice the 582 green tests (including Integration tests) and 2 changes made by Sagie since the last build.

* Going to NUnit Details, you can get the full details:

I had to cut the pictures in order to keep a sane width for the post, but you can get the drift.

btw - Aviel, yet another Semingo hacker add a "Doh!" (Simpson) each time the CC is red on his computer. It can be quite funny (and scary, if fully concentrated on code).

A few pearls from the office:

• Pasha: Damn, I like TDD so much. Writing a 5 minutes piece of code takes about a day (sarcastic tone, of course).
  
• Aviel: in my case, code does not come from my head but rather from my heart.
  Pasha: I saw your code. It comes from your ass.
  
• Me: Integration environment is the Tamagochi of an Agile Team
  
• (Phone ringing, Pasha answers)
  Robert: I'm Robert.
  Pasha: I'm Robert.
  Robert: I'm Robert.
  Pasha: I'm not Robert, I'm Pasha.
  Robert: Oh, My bad.
  Pasha: Bye then.
  (end of conversation).

After almost 1.5 months of Scrum at Semingo (a baby startup), I decided to expose the way we work at the moment and talk about the adjustments we've made in order to suit Scrum to our needs.

I'll start with our implementation to the Daily Stand-up Meeting. No doubt, our meetings are pretty funny (most of the time) and create the right vibe for the Team. The one thing I love most about our meetings is that by the end of the meeting, I know what is the general work-plan for each member in my Team.

Now it's easy to know what I'm planning to complete today (I spend 5-10 minutes planning my day before the meeting), when & if I need to finish something earlier (or at least decide about interfaces after the meeting) in order to integrate with others, help out or ask for someone's else help(code review for example), stay after the meeting in order to talk about something that pop up during the meeting, remove impediments (if I can) and most importantly - have a good laugh before the day begins.

Daily Stand-up Meeting (aka DSM) structure at Semingo:

When: 
Every day at 10:30.
Where:
Meetings room.
Time Box:
15 minutes.
Attendants: 
Pigs only (Chickens can (only) listen)
On the table: 
Each Pig answer these 4(!) questions:
(1) What have I done since the last DSM ?
(2) What am I planning to do until the next DSM ?
(3) Impediments - what bothers me to work ?
(4) Am I on track?
     If someone feels that one of his tasks won't be finished as planned, a flag is raised so the Team could assist. 
     The same goes if someone feels that he's going to finish before the expected time. 
     This means that he could help out someone else or take a few extra tasks we did not plan for the current iteration.

Notes:
Only one Pig talks at a time and he leads the conversation if reasonable questions comes up. He (alone) has the power to stop a conversation if he feels the conversation stray from the DSM path. Team members can decide to talk about an issue that was raised during the DSM just after the meeting is finished.

What next (DSM planned improvements)?
(1) You late, you get (punished, that is): each team member that late to the DSM must wear the "I was late to DSM, I will serve you coffee today" sign on his shirt.
(2) Red-Back on track: If the conversation is getting out of control (too many jokes, drill-down conversations, more than 1 Pig talk etc) - the red button is clicked and each Team member is being electrified with 120V. Well no, but it could have been a nice feature right? Clicking the red button will make a nice GONG!! so we could move on. The Team agree to listen to the GONG and get back on track, so we could finish in time and keeping the DSM productive.

Gosh, I did not know Raymond Lewallen was reading my blog (I guess I should start writing some meaningful stuff and stop playing around ;)) but I'm more than happy to raise up to the challenge and talk about what I am doing in order to go to the next level.

In one of my post, What it takes to become a great developer, I mentioned the notion of "Be Eager To Learn". I don't consider myself as a good developer due to my natural skills (I don't think that I'm mediocre, but certainly not Larry Page). Starting 8 years ago as a little teenager at 15, I had to work my ass off in order to keep up and show the rest of the people I was working with that I'm just as good as they are. Reaching this goal, I wanted to show myself that I can be the best guy at the company.

Eight years passed and a lot have changed, but I'm still very much eager to get better and more versatile. One thing I'll always keep with me, as it proved it self so far, is the no-fear attitude and the (sometimes) ridiculous optimism. I'm not afraid of doing new things or changing positions when an "offer you can't refuse" knocks on my door. Life is short and you most grow each and every day. I'm still the same team player guy, although I can get over confident (aka arrogant) or raise my voice here and there. I care about my teammates and know when to say "I'm sorry". I work with my heart and hopefully my current and future teammates will forgive me for my faults.

I think that in the last few years I've learned a lot about myself, about the things that really intrigued me, that push me to excel. I love coding, I love talking with people, mentoring, lecturing about technologies or Agile methodologies, but most of all - I enjoy taking ownership of projects I participate in and making them successful. I'm looking to surround myself with people smarter than me, those that have natural gifts in them, and making them better.

Things I should do

I should try to get more organized in planning my time. I read a lot of books about self management but I don't feel like I'm practicing them as much as I should. I should really invest more time in myself, trying to set goals and constantly reviewing them. I'm leading the Agile a la Scrum at Semingo so I hope to use this work & review notion more in my life.

I should learn more about Agile, Scrum and XP. I've read a few great books about Agile\Scrum\Management but I still have a lot of unanswered questions. I know that these methodologies only offer some solutions but I don't believe we should enforce them. I believe in making our own Agile process at Semingo. That said, I do want to read more books from people with different experience, different ideas and best practices I could learn from.

I should definitely write more posts! (particularly about Agile\Scrum)

Things I want to do

TDD: getting better in it and start lecturing about it more.
Multi-threading: This one is a new set of skills I'm developing at my current job. Looking at the near future, this skill is crucial as a developer.
WCF: I need to use it in my current job and I have a lot of catch up to do.
Lecturing: At least 4-5 lectures a year looks like a solid goal at the moment.

Most of all, I want to make Semingo the best place to work at, to bring more amazing guys&gals to work with us and making an application that will change the way millions of people work.

Things I won't do

I think that it's getting clear to me that I do not want to be an external coach. I don't see myself coaching a team for a 2-3 months and then shifting to another team. I enjoy working with people and I take pride and strength in making things complete.

I won't stop talking and writing about software, practices and people as long as I have keyboard and working set of 1-N fingers available. Count on it!

Tagging these folks

Pasha Bitz, Shani Raba, Doron Yaacoby, Eran Nachum, Ken Egozi

One of the downsides of using lock is obviously performance. While locking an object, any other thread trying to acquire the lock on that object will wait in line. This can open up a deep hole to performance hit. Rule of thumb while working with locks is to acquire it as late as possible and release it as soon as possible. To demonstrate the order of magnitude bad usage of locks can affect your performance, I decided to write a little demo. 
So let's assume we have a component that is responsible for executing tasks while getting new ones in the process (on different threads). I tried to make this example as simple as possible. Let's start with our "task" class:

public class Task
{
   private int _id;
   private string _name;

   public Task(int id, string name) {
      _id = id;
      _name = name;
   }

   public int Id { // getter, setter }
   public string Name { // getter, setter }
}

We have a TasksRunner that's responsible for getting new tasks and saving it to internal list and executing the current tasks every X milliseconds (via timer). In order to simulate a real-life process, I've made sure that executing a single task is expensive. Let's start with the non-optimized solution:

public class TasksRunner
{
   private List<Task> _tasks;
   private System.Timers.Timer _handleTasksTimer;

   public TasksRunner()
   {
      _tasks = new List<Task>();

      _handleTasksTimer = new Timer(200); 
      _handleTasksTimer.Elapsed += new System.Timers.ElapsedEventHandler(_handleTasksTimer_Elapsed);
      _handleTasksTimer.Start();
   }

   public void AddTask(Task t)
   {
      lock (_tasks)
      {
         _tasks.Add(t);
         Console.WriteLine("Task added, id: " + t.Id + ", name: " + t.Name);
      }
   }

   //Execute the (delta) tasks in a thread from the ThreadPool
   private void _handleTasksTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
   {
      ExecuteCurrentTasks();
   }

   public void ExecuteCurrentTasks()
   {
      lock (_tasks)
      {
         foreach (Task t in _tasks)
            ExecuteSingleTask(t);
         
         _tasks.Clear();
      }
   }

   private void ExecuteSingleTask(Task t)
   {
      Console.WriteLine("Handling task, id: " + t.Id + ", name: " + t.Name);
      Thread.Sleep(1000); //simulate long run
   }
}

AddTask will acquire the lock on _tasks and add the new task to the list while ExecuteCurrentTasks will acquire the lock (on _tasks) and simulate real execution on the task. Notice that during the execution, calling AddTask will wait until the current execution will be finished. Using Roy's ThreadTester, we can run the following in order to notice the behavior so far:

static void Main(string[] args)
{
   TasksRunner runner = new TasksRunner();
   ThreadTester threadTester = new ThreadTester();
   threadTester.RunBehavior = ThreadRunBehavior.RunUntilAllThreadsFinish;

   Stopwatch watch = new Stopwatch();
   watch.Start();

   int numberOfTasksToCreate = 100;
   threadTester.AddThreadAction(delegate
      {
         for (int j = 0; j < numberOfTasksToCreate; j++) 
         {
            runner.AddTask(new Task(j, "job " + j));
            Thread.Sleep(100);
         }
      });

   threadTester.StartAllThreads(int.MaxValue); //wait, no matter how long

   Console.WriteLine("Total time so far (milliseconds): " + watch.ElapsedMilliseconds);
   Console.WriteLine("Tasks added so far: " + runner.TasksAdded);
   Console.WriteLine("Tasks executed so far: " + runner.TasksExecuted);
   Console.WriteLine("Waiting for tasks to end...");

   while (runner.TasksExecuted < numberOfTasksToCreate)
      Thread.Sleep(1000);

   runner.Shutdown();

   Console.WriteLine("done!");
   Console.WriteLine("Total time so far (milliseconds): " + watch.ElapsedMilliseconds);
   Console.WriteLine("Tasks added so far: " + runner.TasksAdded);
   Console.WriteLine("Tasks executed so far: " + runner.TasksExecuted);
}

Running this test will give us a very poor result for adding & executing 100 tasks takes around ~99 seconds.

No doubt, the lock on _tasks while executing each and every task in the list is too expensive as we're depend on ExecuteSingleTask (which is expensive by itself). This way, each new task we're trying to add must wait until the current execution is finished. An elegant solution to this problem, suggested by my teammate Tomer Gabel, is to use a temporal object to point to the current tasks thus freeing the lock much quicker. So here is an optimized version of ExecuteCurrentTasks:

public void ExecuteCurrentTasks()
{
   List<Task> copyOfTasks = null;
   lock (_tasks)
   {
      copyOfTasks = _tasks;
      _tasks = new List<Task>();
   }

   foreach (Task t in copyOfTasks)
      ExecuteSingleTask(t);
}

This little refactoring give us around ~11 seconds for adding & executing 100 tasks.

Smoking!

In my previous post, "How to mock static class or static member for testing", Eli replied that mocking static data can be easily achieved via TypeMock with no extra work around it. Although Eli raised a valid point while TypeMock is a very strong framework that you should check out, I still believe that designing for testability overcomes the basic need for testing your code. Trying to use some sort of powerful Profiler-based framework in order to mock your "wires" is a great thing, but I would use it only if refactoring the code will take too much time or effort that will beat the value of performing the change.

I would like to make my teammates think toward testability while designing the API.

Breaking your code so it will be easily tested will make things a little bit more complex in terms of dependency injection (object A will need to get IB in its constructor in order to work rather then creating new instance of B inside of it), but the allegedly weakness of breaking your code, making you stray from the pure OOP you've learned in "OOP for dummies" courses at the university, is the exact strength I see in TDD. Designing the code by writing use-cases, playing with the API, breaking classes for Single Responsibility and constant refactoring wins the purpose of testing solely or pure OO code. Don't get me wrong, pure OO is nice on the surface, but don't get fooled by it. You write code to create successful software, not successful practice of old paradigms.

TDD constantly demands to re-design the system and refactor it to fit the needs the system invocate, having the tests will back you up in the path. This way you can follow the guidelines of a sane development process:
1). Design the general architecture (the big picture)
2). Agree on interfaces between components
3). Unit-test user stories (use-cases) and drill down into the design by each test.
4). Build Integration tests to connect between components.

and so on...

Now, if you want to refactor existing code or add new behaviors, it's simple:
1). Refactor the code.
2). See that everything compiles.
3). Run unit & integrations tests.
4). If all is good - check-in your source.

Claiming that TDD-OO is not pure OO is fine by me. I would prefer TDD-OO based code on pure OO any day.