Friday, July 07, 2006

I wish I could make it to Robert Scoble's OffTheGrid campout

I was in the Emigrant Montana area last year. I took a few pictures while I was there.

A room with a view

I stayed just a few miles from the Chico Hot Springs. That was the view I woke up to every morning. If you get a chance you should definitely attend the OffTheGrid campout.

7/7/2006 8:50:05 PM (Central Standard Time, UTC-06:00)  #    Trackback

Why would Yahoo support an open source version of the Google File System?

Yahoo is supporting the development of Hadoop! Hadoop! is an open source project that is working to create a Distributed File System (think Google File System) and an implementation of MapReduce.

I find this effort by Yahoo! to be rather interesting given that platform pieces like GFS, BigTable, MapReduce and Sawzall give Google quite the edge in building mega-scale services and in Greg Linden's words are 'major force multipliers' that enable them to pump out new online services at a rapid pace. I'd expect Google's competitors to build similar systems and keep them close to their chest not give them away. I suspect that the reason Yahoo! is going this route is that they don't have enough folks to build this in-house and have thus collaborated with Hadoop project to get some help. This could potentially backfire since there is nothing stopping small or large competitors from reusing their efforts especially if it uses a traditional Open Source license. [Dare Obasanjo]

Dare is surprised that Yahoo! is working on open source versions of tools that could give them a competitive advantage. I can't tell if he thinks Yahoo! is making a mistake by doing this though. I suspect he does.

That is exactly why I don't trust Microsoft as a platform vendor. If one of Dare's ideas gives MSN a competitive advantage what do you think the odds are that we'll see that idea rolled in to the .NET Framework? I think the odds are close to 0.

There is nothing wrong with that. That doesn't make Dare a bad guy and it doesn't make Microsoft evil. But it does make Microsoft a poor choice as a platform vendor. I'm tired of waiting years to get access to Microsoft's second hand, second best ideas.

So why would Yahoo! do this? Why would they create open source versions of tools that could give them a short-term competitive advantage? I think Joel Spolsky said it best:

Smart companies try to commoditize their products' complements. [Joel Spolsky]

Yahoo! is not in the business of selling software. They sell advertising. Software is one of their biggest complements. The best way to commoditize software is to open source it.

Google could probably gain some competitive advantage by developing their own operating system. But they don't. I wonder why.

7/7/2006 1:37:22 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Thursday, July 06, 2006

Rails doesn't need "Enterprise" features

Dave Thomas asked the Rails community to add "Enterprise" features during his RailsConf 2006 keynote.

I had been disheartened earlier in the day by Dave Thomas's talk (no offense, Dave!) Dave's a great guy and has been great to the Ruby community. And although I agreed with some of what Dave said, I couldn't have disagreed more with his view on changing Rails to play better in legacy environments. I sure was relieved to hear DHH's talk the following night. But I still recommend that you view his keynote, too: see what you think and let us know your thoughts. [Softies On Rails]

I agree with Jeff from Softies On Rails. I respect Dave Thomas a lot, but I did not care for the message he brought to RailsConf. At times his keynote was quite condescending. I don't think Dave meant it that way, but all his talk of "in the real world" probably wasn't received well by the RailsConf audience.

I work in the "real world" that Dave was talking about. It sucks! I can't wait to get out. Rails is a breath of fresh air precisely because it doesn't target the "enterprise". It was built by an agile team to create new agile web applications. If you need to create a new agile web application then Rails is a perfect match. But if you need to create yet another big upfront designed enterprise monstrosity, Rails is probably not going to work for you.

Dave also talked about improving the deployment of Rails applications. This is more applicable to the general Rails community. But I disagree with his idea that developers shouldn't be worried about how the application is going to run. That is a mistake. It sounds good in theory, but it ends up creating a situation where the developers make decisions that make the system almost impossible to maintain in production. For small teams you are much better off requiring the developers to own the entire system. If possible they should be responsible for testing, customer support, operations, design and development. As soon as you relieve them of responsibility in any of these areas you can guarantee they are going to make decisions that make it more difficult to support the application in that area.

I'm glad that the core team is focused on solving their problems not some enterprise's problems. Because their problems are my problems. It is strange to work with a platform that is built by people who actually use the platform to build real applications. I've spent so many years depending on platforms that Microsoft creates but doesn't actually use that I didn't realize how much I was missing.

Like Jeff said, watch Dave's keynote and let us know what you think. I am looking forward to seeing DHH's keynote.

7/6/2006 8:51:51 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Wednesday, July 05, 2006

David Heinemeier Hansson's RailsConf 2006 Keynote should be available any minute now

I almost missed this. The Keynotes from RailsConf 2006 are being published on ScribeMedia's site. DHH's Keynote was scheduled for release today so it should be available any time now. In the mean time you can see Dave Thomas' and Martin Fowler's keynotes.

7/5/2006 9:42:11 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Saturday, June 24, 2006

Congratulations Les Orchard and good luck Joe Gregorio

Les Orchard just got a new job at Yahoo. He sounds completely psyched. He deserves it. I can't wait to hear what new and exciting stuff he ends up doing.

Joe Gregorio is looking for a new gig. If you are looking for a creative, passionate developer you should check out Joe's resume.

I've never met Les or Joe. But I feel like I know them both after following their online adventures for almost 5 years. Hell I even have a DECAFBAD T-Shirt.

Unfortunately now I won't get a chance to meet Les. My family lives in Ann Arbor and I've always planned on giving Les a call the next time I'm in his neck of the woods. I guess now I'll have to go out to San Francisco to meet him.

Les & Joe are both great guys and they deserve the best this life has to offer. Good luck guys!

6/24/2006 9:17:25 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Wednesday, June 21, 2006

Dates loaded from the database are not valid when using validates_date_time with us_date_format

I spent quite a while trying to track down a problem in the Rails validates_date_time plugin. In the end the fix was simple, but since I'm new to Rails and Ruby I assumed I was doing something wrong. I finally decided to dig into the code and tests for validates_date_time to see if I could find a bug.

The Problem: I have a model called Person. The schema looks like so:


ActiveRecord::Schema.define(:version => 1) do

create_table "people", :force => true do |t|
	t.column "name", :string
	t.column "date_of_birth", :datetime
end

end

I set ActiveRecord::Validations::DateTime.us_date_format = true.

I can create a new Person just fine like so:


ruby script/console
>> require 'pp'
=> true
>> p = Person.create(:name => "Test", :date_of_birth => Date.new(1972, 12, 31))
>> p.valid?
=> true

Great. But when I try to load the person from the database there is a problem.


>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> false
>> p.errors.on(:date_of_birth)
=> "is an invalid date"

What happens if I set ActiveRecord::Validations::DateTime.us_date_format = false? Let's see.


>> ActiveRecord::Validations::DateTime.us_date_format = false
=> false
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> true

The problem seems to be related to the use of us_date_format. So what is going on?

The validate_date_time plugin looks at each attribute that is passed to validates_date and looks at the *_before_type_cast version of the attribute. In our case it is looking at p.date_of_birth_before_type_cast. Here's what it sees:


>> ActiveRecord::Validations::DateTime.us_date_format = true
=> true
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.date_of_birth_before_type_cast
=> "1972-12-31"

So our raw value of our date_of_birth is "1972-12-31". That looks perfectly reasonable since it is an ISO date and ISO dates are the best way to represent dates since they are easily parsed. So why is that date considered invalid?

The validate_date_time plugin uses a parse_date method to parse date values. Here's what it looks like:


def parse_date(value)
	raise if value.blank?
	return value if value.is_a?(Date)
	return value.to_date if value.is_a?(Time)
	raise unless value.is_a?(String)
	
	year, month, day = case value.strip
		# 22/1/06 or 22\1\06
		when /^(\d{1,2})[\\\/\.:-](\d{1,2})[\\\/\.:-](\d{2}|\d{4})$/ then [$3, $2, $1]
		# 22 Feb 06 or 1 jun 2001
		when /^(\d{1,2}) (\w{3,9}) (\d{2}|\d{4})$/ then [$3, $2, $1]
		# July 1 2005
		when /^(\w{3,9} (\d{1,2}) (\d{2}|\d{4}))$/ then [$3, $1, $2]
		# 2006-01-01
		when /^(\d{4})-(\d{2})-(\d{2})$/ then [$1, $2, $3]
		# Not a valid date string
		else raise
	end
	
	month, day = day, month if ActiveRecord::Validations::DateTime.us_date_format
	
	Date.new(unambiguous_year(year), month_index(month), day.to_i)
rescue
	raise DateParseError
end

The last when statement is where our ISO date of "1972-12-31" matches. At that point year = "1972", month = "12", and day = "31". But then if us_date_format is true, the value of month and day get swapped. Now year = "1972", month = "31", and day = "12".

But that can't be right. The unit tests for validates_date_time all run clean and the date_test.rb has test cases for us_date_format. So what gives?

Here are the tests to prove it:


def test_us_date_format
	with_us_date_format do
		{'1/31/06'  => '2006-01-31', '2\28\01'  => '2001-02-28',
		'10/10/80' => '1980-10-10', '7\4\1960' => '1960-07-04'}.each do |value, result|
			assert_update_and_equal result, :date_of_birth => value
		end
	end
end

Running the Plugin tests (after configuring a database for the validates_date_time plugin to use) results in this:


c:> rake test:plugins
Started
.....................
Finished in 0.703 seconds.

21 tests, 121 assertions, 0 failures, 0 errors

The problem is that the test_us_date_format is not testing a date like ours. Let's change it to look like this:


def test_us_date_format
	with_us_date_format do
		{'1/31/06'  => '2006-01-31', '2\28\01'  => '2001-02-28',
		'10/10/80' => '1980-10-10', '7\4\1960' => '1960-07-04',
		'1972-12-31' => '1972-12-31'}.each do |value, result|
			assert_update_and_equal result, :date_of_birth => value
		end
	end
end

That says, when use_date_format is true I expect to receive the same value when I update a date_of_birth field with an ISO formatted date of '1972-12-31'. What happens when we run the tests now:


c:> rake test:plugins
Started
........F..F.........
Finished in 0.75 seconds.

1) Failure:
test_us_date_format(DateTest)
	[./vendor/plugins/validates_date_time/test/abstract_unit.rb:44:in `assert_up
date_and_equal'
	./vendor/plugins/validates_date_time/test/date_test.rb:70:in `test_us_date_
format'
	./vendor/plugins/validates_date_time/test/date_test.rb:69:in `test_us_date_
format'
	./vendor/plugins/validates_date_time/test/abstract_unit.rb:65:in `with_us_d
ate_format'
	./vendor/plugins/validates_date_time/test/date_test.rb:66:in `test_us_date_
format']:
{:date_of_birth=>"1972-12-31"} should be valid.
 is not true.

2) Failure:
test_various_formats(DateTimeTest)
	[./vendor/plugins/validates_date_time/test/abstract_unit.rb:50:in `assert_up
date_and_match'
	./vendor/plugins/validates_date_time/test/date_time_test.rb:12:in `test_var
ious_formats'
	./vendor/plugins/validates_date_time/test/date_time_test.rb:11:in `test_var
ious_formats']:
<"Tue Jan 03 19:00:00 Central Standard Time 2006"> expected to be =~
.

21 tests, 114 assertions, 2 failures, 0 errors
rake aborted!
Command failed with status (1): [c:/ruby/bin/ruby -Ilib;test "c:/ruby/lib/r...]

(See full trace by running task with --trace)

The first failure is the one we're interested in. It demonstrates the problem we're seeing. Let's see if we can fix it.

Replace the parse_date method in validates_date_time.rb with this:


def parse_date(value)
	raise if value.blank?
	return value if value.is_a?(Date)
	return value.to_date if value.is_a?(Time)
	raise unless value.is_a?(String)
	
	year, month, day, is_iso = case value.strip
		# 22/1/06 or 22\1\06
		when /^(\d{1,2})[\\\/\.:-](\d{1,2})[\\\/\.:-](\d{2}|\d{4})$/ then [$3, $2, $1]
		# 22 Feb 06 or 1 jun 2001
		when /^(\d{1,2}) (\w{3,9}) (\d{2}|\d{4})$/ then [$3, $2, $1]
		# July 1 2005
		when /^(\w{3,9} (\d{1,2}) (\d{2}|\d{4}))$/ then [$3, $1, $2]
		# 2006-01-01
		when /^(\d{4})-(\d{2})-(\d{2})$/ then [$1, $2, $3, true]
		# Not a valid date string
		else raise
	end
	
	month, day = day, month if !is_iso && ActiveRecord::Validations::DateTime.us_date_format
	
	Date.new(unambiguous_year(year), month_index(month), day.to_i)
rescue
	raise DateParseError
end

Now we're setting a local variable is_iso to true when our date matches the ISO formatted when statement. Now when is_iso is true we can skip the swapping of the month and day values. Let's run some tests and see if this is working:


c:> rake test:plugins
Started
.....................
Finished in 0.688 seconds.

21 tests, 123 assertions, 0 failures, 0 errors

Perfect. Now everything should be working in the console too. Let's check:


>> ActiveRecord::Validations::DateTime.us_date_format = true
=> true
>> p = Person.find_by_name("Test")
=> #"Test", "id"=>"2", "date_of_birth"=>"
1972-12-31"}>
>> p.valid?
=> true

Looks good. I think that should do it. I wonder if this explains why the validates_date_time plugin is only rated 3 out of 5 stars here. It is a great plugin, it just looks like the us_date_format might be a little rough around the edges.

6/21/2006 9:28:16 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Tuesday, June 20, 2006

Using In Memory Database for RubyCLR Tests

Even better than just using a SQLite database like I talked about yesterday, I'd love to use a SQLite 3 in memory database for the RubyCLR tests.

6/20/2006 8:53:46 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Monday, June 19, 2006

RubyCLR Drop 4 is Ready

John has released RubyCLR Drop 4. It is fully hosted on RubyForge now. That is great because the other day I had to use Subversion to get the latest version when I wasn't able to connect to John's site for some reason.

I haven't spent a lot of time with RubyCLR yet, but I have a couple of changes I'd like to make if I have the time. I'm not complaining mind you, John is doing a great job making progress on RubyCLR. Consider this my TODO list of things I'd like to contribute if I have time. And if the Lazy Web beats me to these, I guess that'd be ok too ;-)

  1. I wish the tests didn't depend on SQL Server Express. I'd love to have the option to connect to SQLite file instead. And since neither SQL Server nor SQLite connectivity ships with the One Click Installer it isn't any easier to get one working over the other.
  2. Unless I missed it, there is no schema for the test database included in the source repository. Personally I'd like to see a Schema.rb using ActiveRecord::Schema.define. That way even if the default was to use SQL Server Express I could quickly get rolling with my preferred database engine.
  3. Speaking of making it easier to switch database engines. It be nice if the database connection settings were pulled out into a single YAML file.

Keep up the great work John!

6/19/2006 7:17:45 PM (Central Standard Time, UTC-06:00)  #    Trackback

 Wednesday, May 24, 2006

Can Ruby on Rails migrations be used to help version public Xml APIs?

When I first watched the Ruby on Rails migration screencast the first thing I thought was, wow, that is exactly what I've dreamed of having to solve the problem of versioning an Xml API.

Ok I lie, the first thing I thought was, holy crap, you mean I wouldn't have to maintain a bunch of one way SQL scripts for migrating a database.

I have no clue what it would take to get Migrations working over an Xml document format(s). Maybe ActiveRecord could be made to work over Xml, or maybe the ideas behind Migrations could be applied some other way. Eventually I want to dig into that. But maybe the lazyweb will beat me to it.

5/24/2006 8:18:30 AM (Central Standard Time, UTC-06:00)  #    Trackback

 Tuesday, May 23, 2006

Top 13 reasons NOT to consider the Microsoft platform for Web 2.0 development

Some background: My frustration with Microsoft has grown over the last several years. Now that I have my own web project my perspective has changed drastically. I am still an ASP.NET developer in my day job, and that probably won't change in the immediate future, but I am now looking for alternatives to Microsoft's platform for my personal projects. From what I've seen so far Ruby on Rails looks very good.

I saw this and it really helped focus some of my recent thinking about Microsoft & their web development platform. I thought it'd be interesting to compare the point of view of a LAMP developer to my point of view, someone who has considered himself a Microsoft developer for 13 years or more.

1. Free Developer Tools

Visual Studio 2005 is great. I'm glad they decided to make it free forever (at least for the lifecycle of VS 2005, who knows what stupid ass decision the Microsoft marketing boobs will make for Orcas)

2. Free Database

You get what you pay for with SQL Express. Just installing it is a nightmare. The upgrade instructions here should be warning enough. I'm still waiting for a clean XCopyable embedded database engine for SQL Server MDF files. I suspect hell will freeze over before I get one.

Meanwhile the idea that you'll be able to move from SQL Express -> SQL Server when you need to scale sounds good. But there are very real limits to how high you can scale a database system vertically. The problem with believing that you can scale from SQL Express to SQL Server and that SQL Server will meet all your future needs is that it lulls you in to baking SQL Server specific features into your system. You'd be better off starting with SQLite knowing that your database is going to change when you need to scale.

3. Microsoft Atlas makes AJAX easier

Atlas looks promising. It is what they should've shipped with ASP.NET 1.0. The fact that they've made so much progress so quickly with Atlas just reinforces my feeling that they deliberately short-changed the client-side features of ASP.NET 1.0.

I can't help but feel that Microsoft is just waiting for WPF to take off so they can let ASP.NET die. I was shocked to see how much work they did in ASP.NET 2.0. It looks like ASP.NET still has a few years left in it. But don't believe for a minute that Microsoft won't abort ASP.NET the minute WPF starts to gain traction. In the end it still comes down to selling Windows & Office licenses. Unless Microsoft can replace that revenue from its online businesses they shouldn't be trusted as a long term vendor of Web Platforms.

4. Microsoft doesn't HAVE to get ALL of your business.

No but they still want it. Don't ever forget that.

Maybe Atlas is Open Source, I haven't paid enough attention to the license...but I kind of doubt they'll be taking patches from the community.

While Atlas doesn't force you to use any other Microsoft Technology that doesn't mean they won't move it in that direction in the future. I have this sneaking suspicion that Atlas is a stop-gap until they can get WPF/E widely deployed. As soon as that happens I think you'll start to see the switch of this particular bait and switch strategy. Don't get me wrong. I hope something like WPF/E gains traction in the years to come. I just don't trust Microsoft to deliver it in a way that I can build a business on.

5. Microsoft solutions can scale.

Duh! I laugh every time I see this SUN/Oracle propaganda regurgitated. But guess what, LAMP scales too. Just ask Google & Amazon.

I recently watched the MySpace session from Mix06. For me it just reinforced the idea that you won't be able to scale the database vertically. Again, I already knew this from personal experience. But my experience is limited to the challenges that MySpace faced when they went from a single database server to multiple database servers. It was great to hear how they had to evolve their backend as they grew.

I imagine most of MySpace's licensing costs are in SQL Server licenses. I wonder how much their code was coupled to SQL Server and how much that influenced their decision to stay with SQL Server. Their existing teams already knew SQL Server and making a switch to some other system would've caused a lot of pain all the way around. But if you don't have to face rewriting code that is working and tested then the pain of retraining employees may not be as bad. Just one more reason why you should make your code database agnostic from the beginning. I used to laugh when I heard people say that. Now I take it very seriously. And I wonder how much this played into Microsoft's decision to go from the abstract database agnostic world view in ADO to the provider specific model in ADO.NET. Maybe it was just a coincidence, but maybe not. The fact that I wonder about such things reinforces how little I really trust Microsoft with my platform decisions.

6. Microsoft pricing is flexible.

Software Assurance. Burn me once...'Nuff said.

7. Ray Ozzie

Can't argue with this. I love Ray. I hope Ray lasts for a very long time at Microsoft. But if he leaves in the next year or so that will be very telling.

8. Robert Scoble

The recent openness of Microsoft is a double-edged sword. It used to be easy to believe that Microsoft was living in a reality distortion field where they all believed the same crap that the corporate marketing machine spews. Now I see that these guys and gals are smart, and they understand a lot of the issues that independent developers face. Yet Microsoft as a company still makes decisions that only serve their self interest. That is a reasonable thing to do for a corporation, but it treats the technology industry as a zero sum game. Google on the other hand seems to understand that everyone can win. They can earn money while helping partners earn money while helping advertisers earn money while helping users get value. I'm more likely to trust platforms that Google builds because our goals are aligned. Microsoft's goals aren't aligned with mine. The bigger my business gets, the more money Microsoft gets to extract from my business in licensing fees without providing any additional value to my business. I don't see this as evil vs. non-evil. I see it as evolution. And right now Google has a tremendous Darwinian advantage. It isn't too late for Microsoft but that time is quickly approaching.

9. Being based on the Microsoft platform doesn’t limit your acquisition options.

Stop worrying about exit strategies and build a good business. That is the sort of thinking that caused the first bubble.

10. Microsoft wants to be a part of the community

They definitely want the advantages of community. I just don't know if they get that real open source communities are ecosystems. I used to think that companies like Google & Amazon are just exploiting the labor of poor naive open source developers. I now see that most open source contributors do so because they get benefits by having the software they depend on maintained by a community. You'd think I'd get that since I've created open source software myself. I guess I didn't make the connection because my reasons weren’t driven by economics. But many open source contributors are driven by economics. It makes a lot of sense for Google to promote Firefox. Their business depends on a solid cross-platform browser that real people can use. I wish they'd do more direct development of Firefox. I still secretly hope that Google is working on an open cross-platform answer to Avalon.

11. Microsoft employees aren't evil.

I find the whole evil/non-evil discussion pointless and immature. In fact I used to be turned off by Google's whole do no evil mantra. I guess that is my own personal hang-up and it is probably because of the way I define good and evil. I prefer to think in terms of how well a company's incentives are aligned to mine. As I said, Microsoft still doesn't compare favorably to companies like Google. It isn't because they are inherently evil or non-evil. It is just because their natural corporate greed is aligned better with the natural greed of many of the other participants in their ecosystem. It's called synergy. Until Microsoft figures out a way to align its goals with mine - as a user, developer, and web based business owner - I can't trust them to do the right things for me.

12. Microsoft has good development resources

Sorry, I've had to suffer under the tyranny of MSDN for 13 years now. There is a lot of STUFF, but not much of it is useful to experienced developers. Just try to get started with Atlas. You will quickly find yourself digging through the code. And every experienced .NET developer keeps Reflector close at hand. There is a reason for that, the documentation sucks.

13. Microsoft speeds web application development

Nope. The demos look good, but ASP.NET sucks as a platform for web development. They made some very bad choices in ASP.NET 1.0. They tried to make web development like VB6 windows development. They wanted to keep developers in their comfort zone so they could easily switch back to doing Windows only development when Microsoft was able to deliver their next generation platform. It might have worked if it didn't take Microsoft 5+ years to deliver that platform. Hell I was lulled into waiting a very long time before really embracing the web as a platform. I really wanted WinForms to work. But again Microsoft killed WinForms before it was even born.

Conclusion

Many people predicted that Microsoft would completely abandon IE once they killed off Netscape. I didn't believe it at the time, but I do now. ASP.NET was a defensive move. Atlas is a defensive move. WPF/E is a defensive move. The long term strategy still depends largely on selling licenses of Windows and Office. They say they now get the web and that they are going to focus on Services. I say bullshit. I'll believe it when I see it. And I am done trusting Microsoft to do the right thing for me until I see them deriving most of their revenue from win-win-win sources.

Take the Mappoint web services as an example. Microsoft built it and the first thing they seemed to think was, how do we charge for this? How can we get our customers to let us hook them up to the Mappoint IV so we can charge them for each drip? Google looked at maps and they seemed to think, how can we help users find stuff easier while giving small advertisers a better way to advertise to their targeted customers and how can we make money doing that? It is not a question of good and evil. It is question of focus. Do you see the web as a zero sum game? If not then be wary of Microsoft because they apparently still do.

5/23/2006 10:18:47 AM (Central Standard Time, UTC-06:00)  #    Trackback


Links

Categories

Archives