Saturday, June 24, 2006

Congratulations Les Orchard and good luck Joe Gregorio

Les Orchard just got a new job at Yahoo. He sounds completely psyched. He deserves it. I can't wait to hear what new and exciting stuff he ends up doing.

Joe Gregorio is looking for a new gig. If you are looking for a creative, passionate developer you should check out Joe's resume.

I've never met Les or Joe. But I feel like I know them both after following their online adventures for almost 5 years. Hell I even have a DECAFBAD T-Shirt.

Unfortunately now I won't get a chance to meet Les. My family lives in Ann Arbor and I've always planned on giving Les a call the next time I'm in his neck of the woods. I guess now I'll have to go out to San Francisco to meet him.

Les & Joe are both great guys and they deserve the best this life has to offer. Good luck guys!

6/24/2006 9:17:25 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Wednesday, June 21, 2006

Dates loaded from the database are not valid when using validates_date_time with us_date_format

I spent quite a while trying to track down a problem in the Rails validates_date_time plugin. In the end the fix was simple, but since I'm new to Rails and Ruby I assumed I was doing something wrong. I finally decided to dig into the code and tests for validates_date_time to see if I could find a bug.

The Problem: I have a model called Person. The schema looks like so:


ActiveRecord::Schema.define(:version => 1) do

create_table "people", :force => true do |t|
	t.column "name", :string
	t.column "date_of_birth", :datetime
end

end

I set ActiveRecord::Validations::DateTime.us_date_format = true.

I can create a new Person just fine like so:


ruby script/console
>> require 'pp'
=> true
>> p = Person.create(:name => "Test", :date_of_birth => Date.new(1972, 12, 31))
>> p.valid?
=> true

Great. But when I try to load the person from the database there is a problem.


>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> false
>> p.errors.on(:date_of_birth)
=> "is an invalid date"

What happens if I set ActiveRecord::Validations::DateTime.us_date_format = false? Let's see.


>> ActiveRecord::Validations::DateTime.us_date_format = false
=> false
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> true

The problem seems to be related to the use of us_date_format. So what is going on?

The validate_date_time plugin looks at each attribute that is passed to validates_date and looks at the *_before_type_cast version of the attribute. In our case it is looking at p.date_of_birth_before_type_cast. Here's what it sees:


>> ActiveRecord::Validations::DateTime.us_date_format = true
=> true
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.date_of_birth_before_type_cast
=> "1972-12-31"

So our raw value of our date_of_birth is "1972-12-31". That looks perfectly reasonable since it is an ISO date and ISO dates are the best way to represent dates since they are easily parsed. So why is that date considered invalid?

The validate_date_time plugin uses a parse_date method to parse date values. Here's what it looks like:


def parse_date(value)
	raise if value.blank?
	return value if value.is_a?(Date)
	return value.to_date if value.is_a?(Time)
	raise unless value.is_a?(String)
	
	year, month, day = case value.strip
		# 22/1/06 or 22\1\06
		when /^(\d{1,2})[\\\/\.:-](\d{1,2})[\\\/\.:-](\d{2}|\d{4})$/ then [$3, $2, $1]
		# 22 Feb 06 or 1 jun 2001
		when /^(\d{1,2}) (\w{3,9}) (\d{2}|\d{4})$/ then [$3, $2, $1]
		# July 1 2005
		when /^(\w{3,9} (\d{1,2}) (\d{2}|\d{4}))$/ then [$3, $1, $2]
		# 2006-01-01
		when /^(\d{4})-(\d{2})-(\d{2})$/ then [$1, $2, $3]
		# Not a valid date string
		else raise
	end
	
	month, day = day, month if ActiveRecord::Validations::DateTime.us_date_format
	
	Date.new(unambiguous_year(year), month_index(month), day.to_i)
rescue
	raise DateParseError
end

The last when statement is where our ISO date of "1972-12-31" matches. At that point year = "1972", month = "12", and day = "31". But then if us_date_format is true, the value of month and day get swapped. Now year = "1972", month = "31", and day = "12".

But that can't be right. The unit tests for validates_date_time all run clean and the date_test.rb has test cases for us_date_format. So what gives?

Here are the tests to prove it:


def test_us_date_format
	with_us_date_format do
		{'1/31/06'  => '2006-01-31', '2\28\01'  => '2001-02-28',
		'10/10/80' => '1980-10-10', '7\4\1960' => '1960-07-04'}.each do |value, result|
			assert_update_and_equal result, :date_of_birth => value
		end
	end
end

Running the Plugin tests (after configuring a database for the validates_date_time plugin to use) results in this:


c:> rake test:plugins
Started
.....................
Finished in 0.703 seconds.

21 tests, 121 assertions, 0 failures, 0 errors

The problem is that the test_us_date_format is not testing a date like ours. Let's change it to look like this:


def test_us_date_format
	with_us_date_format do
		{'1/31/06'  => '2006-01-31', '2\28\01'  => '2001-02-28',
		'10/10/80' => '1980-10-10', '7\4\1960' => '1960-07-04',
		'1972-12-31' => '1972-12-31'}.each do |value, result|
			assert_update_and_equal result, :date_of_birth => value
		end
	end
end

That says, when use_date_format is true I expect to receive the same value when I update a date_of_birth field with an ISO formatted date of '1972-12-31'. What happens when we run the tests now:


c:> rake test:plugins
Started
........F..F.........
Finished in 0.75 seconds.

1) Failure:
test_us_date_format(DateTest)
	[./vendor/plugins/validates_date_time/test/abstract_unit.rb:44:in `assert_up
date_and_equal'
	./vendor/plugins/validates_date_time/test/date_test.rb:70:in `test_us_date_
format'
	./vendor/plugins/validates_date_time/test/date_test.rb:69:in `test_us_date_
format'
	./vendor/plugins/validates_date_time/test/abstract_unit.rb:65:in `with_us_d
ate_format'
	./vendor/plugins/validates_date_time/test/date_test.rb:66:in `test_us_date_
format']:
{:date_of_birth=>"1972-12-31"} should be valid.
 is not true.

2) Failure:
test_various_formats(DateTimeTest)
	[./vendor/plugins/validates_date_time/test/abstract_unit.rb:50:in `assert_up
date_and_match'
	./vendor/plugins/validates_date_time/test/date_time_test.rb:12:in `test_var
ious_formats'
	./vendor/plugins/validates_date_time/test/date_time_test.rb:11:in `test_var
ious_formats']:
<"Tue Jan 03 19:00:00 Central Standard Time 2006"> expected to be =~
.

21 tests, 114 assertions, 2 failures, 0 errors
rake aborted!
Command failed with status (1): [c:/ruby/bin/ruby -Ilib;test "c:/ruby/lib/r...]

(See full trace by running task with --trace)

The first failure is the one we're interested in. It demonstrates the problem we're seeing. Let's see if we can fix it.

Replace the parse_date method in validates_date_time.rb with this:


def parse_date(value)
	raise if value.blank?
	return value if value.is_a?(Date)
	return value.to_date if value.is_a?(Time)
	raise unless value.is_a?(String)
	
	year, month, day, is_iso = case value.strip
		# 22/1/06 or 22\1\06
		when /^(\d{1,2})[\\\/\.:-](\d{1,2})[\\\/\.:-](\d{2}|\d{4})$/ then [$3, $2, $1]
		# 22 Feb 06 or 1 jun 2001
		when /^(\d{1,2}) (\w{3,9}) (\d{2}|\d{4})$/ then [$3, $2, $1]
		# July 1 2005
		when /^(\w{3,9} (\d{1,2}) (\d{2}|\d{4}))$/ then [$3, $1, $2]
		# 2006-01-01
		when /^(\d{4})-(\d{2})-(\d{2})$/ then [$1, $2, $3, true]
		# Not a valid date string
		else raise
	end
	
	month, day = day, month if !is_iso && ActiveRecord::Validations::DateTime.us_date_format
	
	Date.new(unambiguous_year(year), month_index(month), day.to_i)
rescue
	raise DateParseError
end

Now we're setting a local variable is_iso to true when our date matches the ISO formatted when statement. Now when is_iso is true we can skip the swapping of the month and day values. Let's run some tests and see if this is working:


c:> rake test:plugins
Started
.....................
Finished in 0.688 seconds.

21 tests, 123 assertions, 0 failures, 0 errors

Perfect. Now everything should be working in the console too. Let's check:


>> ActiveRecord::Validations::DateTime.us_date_format = true
=> true
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> true

Looks good. I think that should do it. I wonder if this explains why the validates_date_time plugin is only rated 3 out of 5 stars here. It is a great plugin, it just looks like the us_date_format might be a little rough around the edges.

6/21/2006 9:28:16 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Tuesday, June 20, 2006

Using In Memory Database for RubyCLR Tests

Even better than just using a SQLite database like I talked about yesterday, I'd love to use a SQLite 3 in memory database for the RubyCLR tests.

6/20/2006 8:53:46 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Monday, June 19, 2006

RubyCLR Drop 4 is Ready

John has released RubyCLR Drop 4. It is fully hosted on RubyForge now. That is great because the other day I had to use Subversion to get the latest version when I wasn't able to connect to John's site for some reason.

I haven't spent a lot of time with RubyCLR yet, but I have a couple of changes I'd like to make if I have the time. I'm not complaining mind you, John is doing a great job making progress on RubyCLR. Consider this my TODO list of things I'd like to contribute if I have time. And if the Lazy Web beats me to these, I guess that'd be ok too ;-)

  1. I wish the tests didn't depend on SQL Server Express. I'd love to have the option to connect to SQLite file instead. And since neither SQL Server nor SQLite connectivity ships with the One Click Installer it isn't any easier to get one working over the other.
  2. Unless I missed it, there is no schema for the test database included in the source repository. Personally I'd like to see a Schema.rb using ActiveRecord::Schema.define. That way even if the default was to use SQL Server Express I could quickly get rolling with my preferred database engine.
  3. Speaking of making it easier to switch database engines. It be nice if the database connection settings were pulled out into a single YAML file.

Keep up the great work John!

6/19/2006 7:17:45 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Wednesday, May 24, 2006

Can Ruby on Rails migrations be used to help version public Xml APIs?

When I first watched the Ruby on Rails migration screencast the first thing I thought was, wow, that is exactly what I've dreamed of having to solve the problem of versioning an Xml API.

Ok I lie, the first thing I thought was, holy crap, you mean I wouldn't have to maintain a bunch of one way SQL scripts for migrating a database.

I have no clue what it would take to get Migrations working over an Xml document format(s). Maybe ActiveRecord could be made to work over Xml, or maybe the ideas behind Migrations could be applied some other way. Eventually I want to dig into that. But maybe the lazyweb will beat me to it.

5/24/2006 8:18:30 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Tuesday, May 23, 2006

Top 13 reasons NOT to consider the Microsoft platform for Web 2.0 development

Some background: My frustration with Microsoft has grown over the last several years. Now that I have my own web project my perspective has changed drastically. I am still an ASP.NET developer in my day job, and that probably won't change in the immediate future, but I am now looking for alternatives to Microsoft's platform for my personal projects. From what I've seen so far Ruby on Rails looks very good.

I saw this and it really helped focus some of my recent thinking about Microsoft & their web development platform. I thought it'd be interesting to compare the point of view of a LAMP developer to my point of view, someone who has considered himself a Microsoft developer for 13 years or more.

1. Free Developer Tools

Visual Studio 2005 is great. I'm glad they decided to make it free forever (at least for the lifecycle of VS 2005, who knows what stupid ass decision the Microsoft marketing boobs will make for Orcas)

2. Free Database

You get what you pay for with SQL Express. Just installing it is a nightmare. The upgrade instructions here should be warning enough. I'm still waiting for a clean XCopyable embedded database engine for SQL Server MDF files. I suspect hell will freeze over before I get one.

Meanwhile the idea that you'll be able to move from SQL Express -> SQL Server when you need to scale sounds good. But there are very real limits to how high you can scale a database system vertically. The problem with believing that you can scale from SQL Express to SQL Server and that SQL Server will meet all your future needs is that it lulls you in to baking SQL Server specific features into your system. You'd be better off starting with SQLite knowing that your database is going to change when you need to scale.

3. Microsoft Atlas makes AJAX easier

Atlas looks promising. It is what they should've shipped with ASP.NET 1.0. The fact that they've made so much progress so quickly with Atlas just reinforces my feeling that they deliberately short-changed the client-side features of ASP.NET 1.0.

I can't help but feel that Microsoft is just waiting for WPF to take off so they can let ASP.NET die. I was shocked to see how much work they did in ASP.NET 2.0. It looks like ASP.NET still has a few years left in it. But don't believe for a minute that Microsoft won't abort ASP.NET the minute WPF starts to gain traction. In the end it still comes down to selling Windows & Office licenses. Unless Microsoft can replace that revenue from its online businesses they shouldn't be trusted as a long term vendor of Web Platforms.

4. Microsoft doesn't HAVE to get ALL of your business.

No but they still want it. Don't ever forget that.

Maybe Atlas is Open Source, I haven't paid enough attention to the license...but I kind of doubt they'll be taking patches from the community.

While Atlas doesn't force you to use any other Microsoft Technology that doesn't mean they won't move it in that direction in the future. I have this sneaking suspicion that Atlas is a stop-gap until they can get WPF/E widely deployed. As soon as that happens I think you'll start to see the switch of this particular bait and switch strategy. Don't get me wrong. I hope something like WPF/E gains traction in the years to come. I just don't trust Microsoft to deliver it in a way that I can build a business on.

5. Microsoft solutions can scale.

Duh! I laugh every time I see this SUN/Oracle propaganda regurgitated. But guess what, LAMP scales too. Just ask Google & Amazon.

I recently watched the MySpace session from Mix06. For me it just reinforced the idea that you won't be able to scale the database vertically. Again, I already knew this from personal experience. But my experience is limited to the challenges that MySpace faced when they went from a single database server to multiple database servers. It was great to hear how they had to evolve their backend as they grew.

I imagine most of MySpace's licensing costs are in SQL Server licenses. I wonder how much their code was coupled to SQL Server and how much that influenced their decision to stay with SQL Server. Their existing teams already knew SQL Server and making a switch to some other system would've caused a lot of pain all the way around. But if you don't have to face rewriting code that is working and tested then the pain of retraining employees may not be as bad. Just one more reason why you should make your code database agnostic from the beginning. I used to laugh when I heard people say that. Now I take it very seriously. And I wonder how much this played into Microsoft's decision to go from the abstract database agnostic world view in ADO to the provider specific model in ADO.NET. Maybe it was just a coincidence, but maybe not. The fact that I wonder about such things reinforces how little I really trust Microsoft with my platform decisions.

6. Microsoft pricing is flexible.

Software Assurance. Burn me once...'Nuff said.

7. Ray Ozzie

Can't argue with this. I love Ray. I hope Ray lasts for a very long time at Microsoft. But if he leaves in the next year or so that will be very telling.

8. Robert Scoble

The recent openness of Microsoft is a double-edged sword. It used to be easy to believe that Microsoft was living in a reality distortion field where they all believed the same crap that the corporate marketing machine spews. Now I see that these guys and gals are smart, and they understand a lot of the issues that independent developers face. Yet Microsoft as a company still makes decisions that only serve their self interest. That is a reasonable thing to do for a corporation, but it treats the technology industry as a zero sum game. Google on the other hand seems to understand that everyone can win. They can earn money while helping partners earn money while helping advertisers earn money while helping users get value. I'm more likely to trust platforms that Google builds because our goals are aligned. Microsoft's goals aren't aligned with mine. The bigger my business gets, the more money Microsoft gets to extract from my business in licensing fees without providing any additional value to my business. I don't see this as evil vs. non-evil. I see it as evolution. And right now Google has a tremendous Darwinian advantage. It isn't too late for Microsoft but that time is quickly approaching.

9. Being based on the Microsoft platform doesn’t limit your acquisition options.

Stop worrying about exit strategies and build a good business. That is the sort of thinking that caused the first bubble.

10. Microsoft wants to be a part of the community

They definitely want the advantages of community. I just don't know if they get that real open source communities are ecosystems. I used to think that companies like Google & Amazon are just exploiting the labor of poor naive open source developers. I now see that most open source contributors do so because they get benefits by having the software they depend on maintained by a community. You'd think I'd get that since I've created open source software myself. I guess I didn't make the connection because my reasons weren’t driven by economics. But many open source contributors are driven by economics. It makes a lot of sense for Google to promote Firefox. Their business depends on a solid cross-platform browser that real people can use. I wish they'd do more direct development of Firefox. I still secretly hope that Google is working on an open cross-platform answer to Avalon.

11. Microsoft employees aren't evil.

I find the whole evil/non-evil discussion pointless and immature. In fact I used to be turned off by Google's whole do no evil mantra. I guess that is my own personal hang-up and it is probably because of the way I define good and evil. I prefer to think in terms of how well a company's incentives are aligned to mine. As I said, Microsoft still doesn't compare favorably to companies like Google. It isn't because they are inherently evil or non-evil. It is just because their natural corporate greed is aligned better with the natural greed of many of the other participants in their ecosystem. It's called synergy. Until Microsoft figures out a way to align its goals with mine - as a user, developer, and web based business owner - I can't trust them to do the right things for me.

12. Microsoft has good development resources

Sorry, I've had to suffer under the tyranny of MSDN for 13 years now. There is a lot of STUFF, but not much of it is useful to experienced developers. Just try to get started with Atlas. You will quickly find yourself digging through the code. And every experienced .NET developer keeps Reflector close at hand. There is a reason for that, the documentation sucks.

13. Microsoft speeds web application development

Nope. The demos look good, but ASP.NET sucks as a platform for web development. They made some very bad choices in ASP.NET 1.0. They tried to make web development like VB6 windows development. They wanted to keep developers in their comfort zone so they could easily switch back to doing Windows only development when Microsoft was able to deliver their next generation platform. It might have worked if it didn't take Microsoft 5+ years to deliver that platform. Hell I was lulled into waiting a very long time before really embracing the web as a platform. I really wanted WinForms to work. But again Microsoft killed WinForms before it was even born.

Conclusion

Many people predicted that Microsoft would completely abandon IE once they killed off Netscape. I didn't believe it at the time, but I do now. ASP.NET was a defensive move. Atlas is a defensive move. WPF/E is a defensive move. The long term strategy still depends largely on selling licenses of Windows and Office. They say they now get the web and that they are going to focus on Services. I say bullshit. I'll believe it when I see it. And I am done trusting Microsoft to do the right thing for me until I see them deriving most of their revenue from win-win-win sources.

Take the Mappoint web services as an example. Microsoft built it and the first thing they seemed to think was, how do we charge for this? How can we get our customers to let us hook them up to the Mappoint IV so we can charge them for each drip? Google looked at maps and they seemed to think, how can we help users find stuff easier while giving small advertisers a better way to advertise to their targeted customers and how can we make money doing that? It is not a question of good and evil. It is question of focus. Do you see the web as a zero sum game? If not then be wary of Microsoft because they apparently still do.

5/23/2006 10:18:47 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Monday, February 06, 2006

Progress Report: I lost 4.6 pounds last week and 20.9 pounds in January 2006

I lost 4.6 pounds last week and 20.9 pounds in January 2006.

If you are curious you can read more on the PhysicsDiet.com Blog.

2/6/2006 10:00:21 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Monday, January 30, 2006

Diet Progress: I lost another 4.4 pounds last week

For now on I will be posting my diet progress and other diet related posts on the PhysicsDiet.com Blog. You can see my latest progress report here: Diet Progress: I lost another 4.4 pounds last week

1/30/2006 9:31:00 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Sunday, January 22, 2006

Losing Weight

In October I read The Hacker's Diet. I first heard of The Hacker's Diet years ago but I thought it was a joke. Hackers live on pizza and Coke after all.

Two ideas in The Hacker's Diet really stuck with me. The first was using a moving average to track my weight loss. The second was the idea of using frozen dinners to control portion sizes and to count calories. The idea of counting calories never appealed to me because I couldn't imagine myself measuring and weighing all the food I eat. But I could definitely see myself eating just the controlled portions in a frozen dinner. Especially if the frozen dinners were good.

I bought a scale and started my diet on 10/28/2005. At that time I weighed 324 pounds. At a height of 6' 2" that put my Body Mass Index at 41.6. At the time I hadn't calculated my BMI so I didn't realize that I was officially morbidly obese. I'm not sure which is more embarrassing, that I was morbidly obese, or that I didn't realize that I was morbidly obese.

I started weighing myself and recoding my weights every day. At first I tried to use The Hacker's Diet Spreadsheets. But those spreadsheets haven't aged well and they don't work well in newer versions of Excel. So I created my own software to track my diet. It didn't take long for me to realize that the software I created for myself could be useful to a lot of people. So I started to tune the software so I could make it available to anyone who wants to use it. And that is why I'm posting about my today. I'm finally ready to make my software public.

If you'd like to lose a little or a lot of weight, I highly recommend The Hacker's Diet. And if you decide to give The Hacker's Diet a try, I invite you to use my new site PhysicsDiet.com to track your diet. PhysicsDiet.com is free now and I plan to keep it free forever. I plan to support it with income from non-intrusive advertising like Google Adsense and Amazon Associate links.

This morning I weighed 265 pounds. In just under 3 months I have lost 59 pounds. Below is what my current PhysicsDiet.com weight chart looks like.

My PhysicsDiet.com Weight Chart from January 22, 2006

I want to lose another 60 pounds or so in the next 3-4 months. I still have a lot work to do on PhysicsDiet.com, but I plan to do a lot of that work in the next 3-4 months too. As I lose the weight that I've gained over the last 8 years, I feel like I'm getting my life back. A few months ago I posted about Making Meaning. I think I finally found a way to do that through software.

I will post more about my diet and PhysicsDiet.com in the coming weeks and months.

1/22/2006 7:51:40 PM (Central Standard Time, UTC-06:00)  #    Trackback


Links

Categories

Archives