Today I sat with Moran after he "paged" me. He reviewed some code of one of our applications and he saw some things he thought he could make better. It was one of those classic "Hey! it should be a single service which every one of our applications will use!". Step after step he made the required refactoring and some elegant API took shape. The improvement was in magnitude but I still thought that the API should be quite different before making it "public" and placing it in our infrastructure. But, and that's a big but, the guys that develop the specific application needed the class and Moran was required to help in another project.

Should Moran take the opportunity to invest some time in this API? Should I help him through? We can talk about services, about providers, about extendability. We can play with code just to see how the API will look like. We can play with ideas, learn from each other, share our experience. We can share with others, we can send some quick "API tests" to get some feedbacks.

The process of developing the basics for our applications should be *long*. It should allow space for errors. The margins should be wide enough to let us experience, to learn from our mistakes, to discuss, to make Design Reviews, To Prototyping and throwing it all to the garbage 2 days later. It's all OK.

But maybe this is not the right time for games. Maybe Moran should make it work somehow and carry on to the next project ? After all, the next project's deadline is near(like always) and we need to join forces just to keep it up. Hey! we get paid to reach deadlines, to make it happen while keeping the quality at high level. The deadline is sacred. I truly believe that a good team will deliver on time even on the expense of features or well-known but low prioritized bugs.

On the one hand, I know that developing small to medium applications (0.5-2 human years) almost never allow you to invest the required amount of time in the basics. On the other hand, I also know that developing applications at work and infrastructures at home is not the answer for long terms. The process is short. No errors are allowed. There is almost no Design Reviews. Does it mean that the results of "home infrastructures" will be poor ? of course not, but the experience, the ability to break our requirements to programmer stories, to exchange ideas, to grow - is lost. The funniest part is that developing small application without a solid infrastructure turns into medium-large application. I guess it's the egg-chicken paradox.

The best I can do is to tell the reality from my perspective. My reality is that there is never time(well, that depends on the urgency and the magnitude of the application), and keeping with my expectations makes me perform the global thinking and implementation at home. This is the only time that I actually "have the time". Investing my time in developing some required infrastructure can save my guys at work a significant amount of work. Still, putting the effort on developing infrastructure during work hours will come on the expense of developing user stories, mentoring, guiding, consulting, talking with our customers. I guess that I still don't know my place. I'm a good developer, I would like to think, and coding some really interesting delegates-based infrastructure or some neat OOP solution are those treats I can't live without. Still, leading projects, make sure everything ticks and the quality is high is a challenge I love to face in my every-day work; Above all - seeing my guys getting to the next step and helping them in this journey is the main reason I'm doing what I'm doing. I want to be the best I can be for my team and yet make an influence in the way we work via developing some solid infrastructures. My gut feeling is that I need to get better. I feel that I can and should be a lot better as a manager and a programmer but It is still very hard for me to decide how and where to invest my time.

This issue keeps me awake at nights "lately"(last 6 months or so).

Where do you put the line ? How do you decide to invest your time in programming on the expense of managing and vice versa ? When do you think it'ss appropriate to invest additional hour\day\month to something you believe in ? Do you have some rules of thumb ?

I'm proud to introduce "Filter By Expressions". As I mentioned in my previous post about LINQ and IQueryable<T>, I got several ideas about implementing some sort of data(base) filter mechanism just until LINQ will be stable enough. I have  3 alternatives: (1) Implementing some sort of HQL language (like NHibernate) or query parser, (2) use an existing infrastructure and (3) Implement my own mechanism. Considering the options with my managers' hat on, options (1) and (2) are irrelevant at this stage as the learning curve and the adjustments I'll have to make is simply too much for me at the moment. In addition, LINQ will be here shortly so I don't want to invest more than 2 days of work. This leaves me with option (3).

After some good thinking(nothing more) during the weekend and 15 minutes of exchanging ideas with Moran, the ^[1]API took form. Here is a quick example (from one of our .aspx files):

IFilterObject filter = new Filter();
filter.PageResults = true;
filter.CurrentPage = 1;
filter.PageSize = 10;

// Here is the gold !
filter.Where(
   Expression.EqualTo(Order.Field.PurchaseDate, DateTime.Today),
   Expression.Operator(OperatorType.And),
   Expression.EqualTo(Order.Field.IsValid, true)
);

ordersGrid.DataSource = OrdersGateway.Instance.Get(filter);

Any idea what kind of "Where clause" it will generate behind the scenes ?
I bet you do so here it goes(SqlServer syntax):

WHERE Orders.PurchaseDate = @p0 AND Orders.IsValid = @p1

@p0 and @p1 will be filled with DateTime.Today and true respectively.

Here is another example, this time with "Inner condition":

filter.Where(
   Expression.Block(
      Expression.GreaterThan(User.Field.Age, 18),
      Expression.Operator(OperatorType.And),
      Expression.NotEqualTo(User.Field.Email, "")
   )
   Expression.Operator(OperatorType.Or),
   Expression.EqualTo(User.Field.IsGoldMember, true)
);

This will generate the following query(SqlServer syntax):

WHERE (Users.Age>@p0 AND Users.Email != @p1) Or Users.IsGoldMember = @p2

@p0, @p1 and @p2 will be 18, "" and true respectively.

But what about complex scenarios ? My answer is "keep it simple". With this in mind I think that we covered about 98% of every-day queries, which is good enough for me at the moment.

At the moment, our Query Generator Engine only filter the Where clause so you can't do some sort of magic like LINQ or other infrastructures out there allow you to do. Still, this infrastructure is small enough to be easily integrated with almost any existing infrastructure and yet powerful enough to be extremely useful.

Oh, by the way - at current stage, our engine support queries for SqlServer & Oracle databases.

Enough about high-level API. I'll try to explain the Architecture before I'll start showing you the way I decided to sew this engine. Our first (static)class is Expression, which is kind of Factory for expressions:

Factory of expression ah ? what kind of concrete objects I've got there ?
Well, here are few samples: EqualExpression, NotEqualExpression and OperatorExpression

IDataExpression inherit from IExpression and overload GenerateQuery method with two additional parameters: DatabaseMapper and IDataParameterCollection.

The former one(DatabaseMapper) will simply "tell" the Query Engine how to convert User.Field.Age enum into "Users.Age" string and the latter(IDataParameterCollection) will be an empty collection of parameters which every expression can fill according to its' needs.

Eventually, we've got an Engine that receives the IExpression(s) and generates code from them:

Here is the implementation of IQueryCreatorEngine.GenerateWhereClause():

string IQueryCreatorEngine.GenerateWhereClause(IExpression[] expressions, IDataParameterCollection parameters)
{
   IDataExpression wrapper = (IDataExpression)Expression.Block(expressions);
   return wrapper.GenerateQuery(this.Mapper, parameters);
}

Pretty straight forward: wrap the expressions with DataBlockExpression and let him take care of our business. Let's look at our DataBlockExpression.GenerateQuery method:

protected override string GenerateQueryCore(DatabaseMapper mapper, System.Data.IDataParameterCollection parameters)
{
   StringBuilder whereClause = new StringBuilder(this.Expressions.Length * 150);

   whereClause.Append("( ");

   foreach (IExpression exp in this.Expressions)
   {
      IDataExpression dataExpression = exp as IDataExpression;

      if (dataExpression != null)
         whereClause.Append(dataExpression.GenerateQuery(mapper, parameters));
      else
         whereClause.Append(exp.GenerateQuery());
   }

   whereClause.Append(" )");

   return whereClause.ToString();
}

The only thing we have to do now is to supply some sort of method which implement DatabaseMapper (delegate) signature when we initialize the QueryCreatorEngine. Here is a quick example:

QueryCreatorEngine engine = new QueryCreatorEngine(
   delegate(object field)
   {
       User.Field typedObj = field as User.Field;
       if (typedObj != null)
       {
          switch(typedObj)
          {
              case User.Field.Age: return "Users.Age";
              case User.Field.IsGoldMember: return "Users.IsGoldMember";
              case User.Field.Email: return "Users.Email";
              default: 
               throw new ArgumentException("field not supprted"); //whatever
          }
       }
       else
           throw new 
               ArgumentException("field != User.Field type"); //whatever
   });

That's it.

Our mapper is generated by tool (our lovely Code-Agent) and our Filter By Expressions infrastructure is ready to go !

If you are still here, leave me a comment and I'll send you a T-Shirt of "Code don't make me sleepy!".

I'll upload our Data Access infrastructure in a few days so be patient and let me know if this post made you happy.

---

^[1] We did a quick API exam in our department and the results were good.

I'm doing some thinking about our GUI, Data Access Layer, Entities and the way we should combine them. Looking a little further, LINQ is just a matter of time and effort until we'll use this great infrastructure. I've downloaded a few movies about LINQ from Channel 9, and started to glance at my future. If you're an architect, you should really take a look at Chatting about LINQ and ADO.NET Entities. This will give you a better insight about LINQ's Architecture and where you should intervene if needed.

Our department uses a product we developed in-house named Code-Agent which allows us to generate our application based on our ERD. At current stage, we have our own infrastructure for manipulating data between our tiers, but it lacks some abilities which I really want to see in the near future.

This movie made me *think*.

But before I'll carry on with my insights, let me talk about LINQ, IQueryable<T> and lamda expressions; Let's start with a little example:

var results = from order in orders
              where order.Amount > 100
              select order.Name;

results will now contain all the orders with amount > 100.
The CLR transforms this query into method calls:

var results = orders.
                  Where(delegate(Order order) { return order.Amount>100; }).
                  Select(delegate(Order order) { return order.Name; });

Or via Lamda expression (kind of version 2 to anonymous method):

var results = orders.
               Where(order => order.Amount>100).
               Select(order => order.Name);

Seeing this kind of solution immediately made me think about writing our own LINQ wrapper for the meanwhile. Still, I can't work on production code with LINQ as it really premature at current build but I can definitely build some sort of engine which implement the core principles of LINQ and in time remove my implementation, bit after bit. The next step was to look for IQueryable<T>, this baby inherit from IEnumerable<T> and let you do this magic I've demonstrated above (Where, Select, Group By, Order... you name it). I found a great post on this matter at Wayward Blog.

What's the next step you might ask ?
I guess the answer is to emulate IQueryable<T> in our current infrastructure and aim for compatibility and consistency with LINQ.

Here is a quick implementation which pops out of my head:

EntityCollection<Order> orders = 
                  EntityCollection<Order>.
                     Where(delegate(Order o) { return o.Amount > 100; }).
                     Select(delegate(Order o) { return new { o.Name, o.Id }; });

foreach (Order o in orders)
{
    Console.WriteLine("Order id: " + o.Id + " name: " + o.Name);
}

EntityCollection<T> will inherit from IQueryable<T> and implement the basics.

It's time to get into code-till-I'll-bleed mode...

Reading Oren Eini's post & comments about How to tell that this is not production code made me nervous. I don't know why but for 10 long-enough seconds my eyes twinkled and I managed to pull 5 buttons from my keyboard.

Due to the fact that I know Oren's character from talking to him quite a bit after my lecture(delegates&anonymous mothods), I have the feeling that he codes-for-fun instead of coding-to-succeed, and he really pushes it sometimes. I can't say that I'm no sinner, but I think that during working hours, you should use your team's time in a wiser manner.

You should code with purpose. This purpose should be with one stand to your project deadlines, people quality, maintainability, efficiency, company's goals etc. This is code-to-succeed in my book.

I wrote this comment down as I'm currently responsible for several applications with some code that actually made me think that this code which "should never be production" made its way into production. It's even worse, the knowledge of how and why this code behaves the way it does is *lost*. The people who wrote it are no longer here and I'm stuck with code that shouldn't exists. Oh yes, I almost forgot, this code has numerous bugs and yep, I will have to play along with this dark magic voodoo and get it to work.

One sentence (from Eini's comments) really got me anemic for couple of seconds:
"We are going to try to explain it tomorrow to another developer, who is going to maintain it, and we will see if he likes it or not."

This is *unacceptable*. You know that you're doing some dark magic, you know that this is a disaster waiting to happen and yet you move forward. Let's say that this poor fellow got the idea, what happens if he quit and a new programmer will have to take care of *your* code ? Who says that you'll be there to hold his hand ? Who says that he will "like" it ? You must know when to stop...

Wait a minute, hold on, you got me all wrong here !

I'm all about creativity and enthusiasm(!!!!), especially during work time, but the above is just a bad practice and not an excuse. Does that mean you can't code things you know you'll have to change later on or even throw them to the garbage ??
Hell no ! this is exactly why you can prototype your solutions if required. Just don't forget to throw the prototype to the garbage and start all over again with the good insights you have acquired during the process.

We are working pretty tight with our QA department. Yes, we have some well-documented work procedures but some times I feel that they are too well-documented. After all, we are lazy; We want something easy to see, easy to read & easy to do.

I thought it will be better to document our relations (Development team with QA) in a simple flow diagram:
Bug life cycle - in the eyes of a programmer.ppt (55.5 KB)

update:

I've added a step: what should I (the programmer) do if I can't reproduce the bug ?
Thanks Shani.

Maybe it will help you as well.

It's about time.

Our CTO, Roee Daliyot, is finally taking the time to write about the things he loves - GUI programming.

After writing about delegates & anonymous methods via JavaScript, he just released 2 must-see client-side controls for Drag&Drop and Clone&Drag&Drop.

So go on and pay him a visit.

Tell him I sent you.

I need two methods which will bring me some data.

1st scenario: bring the data from the file system.

   Method name: GetData(string directory, string[] files)

2nd scenario: bring the data from a dll (assembly) file. The data is an embedded Resource inside the assembly.

   Method name: GetData(string assemblyName, string prefixPath, string[] files)

They both sit in some Helper (static)class.

Now, I thought about changing the method names into GetDataFromFS and GetDataFromAssembly. The fact that they are both serve the same purpose makes me feel that this should be a simple overload but yet I feel that this is wrong somehow due to the fact that their domain is quite different. If tomorrow I'll add GetData(string webServiceUrl, string[] files) should it be overload as well? I must say I'm not so sure...

What do you think ? overload or naming by domain^[1] ?

---

^[1] - No. I don't want some factory and concrete classes or strategy pattern implementation; This should be a simple utils class.

Well, I really don't like writing posts like "Today I drank some Cola and wrote some pieces of code", but this is just funny.

I checked my Referrers list and I saw some interesting searches which lead to my blog:

1. delegate oren (www.google.com) - 1st place

   I like that phrase. delegate oren. yep, you can ring me any time. honestly!
   I guess that my lecture on delegates & anonymous methods gave me some good google PR.

Before you'll read the second one, sit down and don't hold
any glass or anything sharp in your hands.

 

2. why is teched dying (www.google.com) - 6th place

   Don't ask me why, don't ask me how, Just laugh... That's what I did.

   I have nothing but kind words to say about TechEd and you can even read some great posts here and here.

   Who is the psycho that wrote this phrase ?!?!  

3. teched 2006 party (www.google.com) - 14th place

   That's better. great party !

update:

4. porno oren (http://search.msn.fr/) - 4th place

   Oh-My-God...

5. modal refresh why god why (www.google.com) - 1st place

   Yes god, why ? well, I have a solution for this one.

After receiving several messages from my friends about problems with my blog and their RSS readers I decided today to check that out. I'm using SharpReader as my RSS reader and I've noticed that my posts just don't get syndicated for some unknown reason.

The way I'm posting messages to my (das)Blog is via "Write & Save & Write" system, meaning - I write for 10 minutes, un-check "Publish" & "Syndicate" checkboxes and hit the "Post to Weblog" button. Then I click Edit on the post and continue writing. After I finish writing, I check the "Publish" & "Syndicate" checkboxes and hit the "Post to Weblog".

After playing for 2 minutes with dasBlog and trying to add\edit posts I came to the conclusion: You should *never* uncheck the "Syndicate" checkbox. This isn't reversible for some strange reason so if I want to continue using my "Write & Save & Write" system the only checkbox I should uncheck is "Publish".

Now all my old posts that were un-"Syndicate" for the first time, can not be re-syndicated.

I think that I should let Scott Hanselman know about this feature\bug, what say you ?