Fun with msdeploy, Extended Protection and Windows Authentication

Trying to push a change to our live servers today, we got this wonderful message from msdeploy:

Error: (09/12/2009 16:10:21) An error occurred when the request was processed on the remote computer.

Error: Child object 'extendedProtection' cannot be added to object 'windowsAuthentication'. The 'windowsAuthentication' provider may not support this deployment.

Error count: 1.

We do this several times a day… it worked yesterday, nobody changed anything, and suddenly today it doesn’t work. (Don’t you just love it when that happens?)

Now, msdeploy is wonderful, but has practically no documentation – which means when an error like this happens, you’re on your own.

A bit of Googling and a couple of lucky guesses later, and we worked out what was causing it. “Extended Protection” is apparently some of new fangled security framework that’s included in recent Windows Updates. Our internal servers install Windows updates automatically; our live servers don’t.

In other words – our staging server had quietly upgraded itself in the night to support extended authentication, and was now trying to push that configuration to the live server, which had absolutely no idea what was going on.

Logging on to the live server and installing the outstanding Windows security updates seems to have fixed it.

Axure RP: Lego For Software Designers

Someone asked me on Twitter a little while back:

@dylanbeattie Do u currently use Axure? If so, could u pls tell what size team it's effective with? And, is it really better than pen-paper?
11:38 AM Nov 7th from web in reply to dylanbeattie

The short Twitter answers are “yes”, “we’ve used it effectively with up to 4 people”, and “yes, because I can’t draw” – in that order.

But a slightly more detailed response would probably help.

Once upon a time, prototyping web apps was easy. You’d draw every page, and then use a site map to demonstrate which links went where. Every page was static; nothing moved, there was no Ajax, no infinite scrolling, no drag’n’drop, and most websites were actually about as interactive as a Choose Your Own Adventure novel. Well, those days are gone. People expect more – richer UIs, better responsiveness, less postbacks and waiting around for pages to load – and with libraries like jQuery, there’s really no excuse for not delivering code that satisfies those expectations.

Question is – how do you prototype a rich user interface? How do you draw a picture of something that won’t sit still? For me, that’s where Axure RP comes in. Axure is a “tool for rapidly creating wireframes, prototypes and specifications for applications and web sites”. It’s a commercial product, so it is, alas, not free (although to put it into perspective, it costs less than hiring a .NET developer for one day) – but it is a uniquely powerful and expressive piece of software that I find myself firing up on an almost daily basis.

In everyday use, it’s like a weird cross between Balsamiq, Visual Basic, and Lego.

• Balsamiq, because it’s easy to mock up static user interfaces by dragging buttons, inputs and form elements on to your page.
• Visual Basic, because it’s easy to add behaviour to those elements using click handlers, events and dynamic controls.
• Lego, because it’s intuitive, and it’s fun, and there is no way anyone is going to look at what you’ve done and think the project is finished.

The game Populous was designed using Lego. I grew up with Lego*. From a very early age, I learned to use Lego bricks to express ideas. I knew every single brick I owned. I could demonstrate an idea I’d had for a car, or a spaceship, or a robot, by assembling these reusable components into a prototype with spinning wheels and moving parts and a sense of scale and colour. Working entirely in plastic bricks actually become very liberating, because it stops you worrying about materials and finishes, and allows you to focus entirely on expressing ideas.

Have you ever showed someone a Lego house and had them say “Hey, that looks great! When can we move in?” No. People know a Lego house is not a real house. They appreciate that the point of a Lego - or cardboard, or clay - model is to demonstrate what you’re planning to do, not show off what you’ve already done.

Have you ever showed anyone an HTML mockup of a web app and had them say “Hey, that looks great! When do we launch?” – and then they look horrified when you explain that you haven’t actually started the build yet?

People don’t grok the difference between HTML mockups and completed web apps the way they grok the difference between Lego houses and real ones. I can’t say I blame them. HTML is HTML – whether it was hacked together late last night in Notepad or generated in the cloud by your domain-driven MVC application framework. The difference doesn’t become apparent until they actually start clicking things – by which point it’s too late; you’ve made your first impression (“wow, the new app is done!”) and it’s all downhill from there.

I think the hardest questions in software are “what are we doing?” and “are we done yet?”. I think good prototypes are absolutely instrumental in answering those questions, and any tool that can help us refine those prototypes without falling into the trap of “well, it looks finished” has to be a Good Thing.

* Some people look at how much Lego I still have and conclude that I never grew up at all…

A (Slightly Faster) URL Resolver Module for ASP.NET MVC

Yesterday, I posted some code I’d hacked together as part of an MVC2 demo that would resolve ASP.NET virtual path URLs on the fly as pages were written to the ASP.NET response stream.

Having run some tests on this code in isolation – it’s actually quite nasty. For running ad-hoc demos on your workstation, it’s fine, but the performance hit of decoding the byte array, doing the regex transform and re-encoding it is something like two hundred times slower than a direct stream copy. Not good. There is now a modified version online at Google Code which is quite a bit faster, but there’s still huge scope for improvement. In particular, although it’s using byte comparisons now to work out where the ~/ combination occurs, it’s still falling back to string comparisons every time it finds a tilde to decide whether that tilde needs replacing or not.

These stats were created using a loop that spins up HTML pages of various sizes – one version full of ASP.NET-style tilde paths, one containing no tildes -  and then writes them 100 times to both a normal MemoryStream and a UrlResolverStream in order to calculate the average rendering time. If a page doesn’t contain any tildes at all, performance is 5-6 times slower than the equivalent direct memory copy – i.e. 21ms instead of 4ms. For pages with lots of tildes, the additional string processing hits quite hard and you’re looking at a slowdown factor of around 30-35x.

Tildes?	Page Size	MemoryStream copy (ms)	UrlResolverStream copy (ms)	Ratio
Yes	50Kb	0.05	0.38	7
Yes	101Kb	0.17	2.51	14
Yes	202Kb	0.19	6.05	31
Yes	405Kb	0.37	13.88	37
Yes	809Kb	0.88	29.47	33
Yes	1,619Kb	1.62	60.55	37
Yes	3,238Kb	3.45	123.40	35
No	50Kb	0.02	0.34	17
No	101Kb	0.07	0.66	9
No	202Kb	0.15	1.28	8
No	405Kb	0.48	2.62	5
No	809Kb	0.83	5.26	6
No	1,619Kb	1.66	10.57	6
No	3,237Kb	3.53	21.17	5

It should be possible to make this considerably faster still; since the code basically scans byte arrays, this is one of those areas where using pointer arithmetic could make a huge difference. I’ll dig out my unsafe hat and my pointer-gloves this weekend and see what I can do to it. In the meantime – play with it, experiment with it, but it’s probably a good idea not to let it within a hundred miles of your live servers :)

A URL Resolver Module for ASP.NET MVC

Update: An improved version of this module, along with some performance stats, is available here. The original version posted here was very, very slow. Probably not a good idea to use it for anything. Ever.

One of the few things I actually liked about ASP.NET WebForms was that you could do things like

<a href=”~/my/account.aspx” runat=”server”>My Account</a>

and ASP.NET would magically turn the tilde character (~) into the current relative application root – so you could debug your apps on http://localhost:4567/ and then deploy them to http://www.myserver.com/some/app/, and your links wouldn’t break.

ASP.NET MVC doesn’t like things that are runat=”server” – and with good reason, I think – but this does mean you can end up with rather a lot of calls to ResolveUrl() sprinked throughout your code.

To get around this, I’ve hacked together an HTTP module that basically rewrites the output stream on the fly. It wraps the HTTP output stream (the thing you're writing to when you Response.Write stuff) in a 'smart' stream wrapper, and the magic naively optimistic part looks like this:

public override void Write(byte[] buffer, int offset, int count) {
  if (HttpContext.Current.Handler is System.Web.Mvc.MvcHandler) {
    HttpContext.Current.Trace.Warn("Resolving URLs in output stream...");
    byte[] data = new byte[count];
    Buffer.BlockCopy(buffer, offset, data, 0, count);
    string html = Encoding.ASCII.GetString(data);

    // Don't try and use Regex transformations on your 
    // entire output stream. It is slow. Like, really, really slow.
    // Take a look at this updated version instead.

    var re = new Regex("(?src|href|action)=\"~/", RegexOptions.IgnoreCase | RegexOptions.Compiled | RegexOptions.Multiline | RegexOptions.ExplicitCapture);
    html = re.Replace(html, "${attr}=\"" + VirtualPathUtility.ToAbsolute("~/"));
    data = Encoding.ASCII.GetBytes(html);
    sink.Write(data, 0, html.Length);
    HttpContext.Current.Trace.Warn("Resolved URLs in output stream.");
  } else {
    sink.Write(buffer, offset, count);
  }
}

Basically, it looks for HTML SRC, ACTION and HREF attributes whose value begins with ~/, and replaces the ~ with the application’s virtual path on the fly. I haven’t tested this code for performance, so I don’t know what kind of impact it’ll have on your page response times, This code is something like 200 times slower than a straight stream copy, but it’s running in a couple of demo apps I’m working on and it seems to work pretty nicely.

The full implementation is over on Google Code if you’re interested.

Code from my HTML5 / MVC2 Talk at SkillsMatter

A big thank-you to everyone who came along to my HTML5 and MVC2 talk at SkillsMatter on Monday – and thanks also to SkillsMatter for hosting us! Whilst I’ve done plenty of talking at unconferences and events like BarCamp, this was the first proper full-length technical talk I’ve given, so I’d really appreciate any feedback – especially since we might be doing a re-run in a couple of weeks.

During the talk, I demo’ed a tiny web app – TagAlong - that I’ve built to showcase some of the new features in HTML 5 and ASP.NET MVC preview 2. This is by no means production code – if nothing else, I’m using static List<T>’s instead of having an actual database, so your changes will disappear every time you restart the app – but it should be pretty easy to get it up and running and poke around.

If you’re interested, the code is online at http://code.google.com/p/tagalong – you’ll need MVC 2 Preview 2 installed to run it, but everything else is included.

A couple of other interesting links that I mentioned during the talk:

• The page structure diagrams are taken from A List Apart
• Dean Edward’s work-in-progress Javascript library to emulate HTML5 in downlevel browsers is online at html5-now.appspot.com
• Kevin Roast created the Asteroids demo
• The Fireworks demo was created by Kenneth Kufluk

 

You Forgot to Say the Magic Word…

In Microsoft SQL Server, this query won’t work:

SELECT * FROM ( SELECT * FROM Customer UNION SELECT * FROM Supplier) ORDER BY CompanyName

But – if you ask nicely, it does exactly what you’d expect:

SELECT * FROM ( SELECT * FROM Customer UNION SELECT * FROM Supplier) PLEASE ORDER BY CompanyName

You won’t believe the look on your colleague’s faces when you solve their problem using simple good manners.

(Of course, it actually works because PLEASE in that context just acts as a table-name alias for result of the UNION sub-select, and sub-selects in SQL Server need to have a name. But don't let that stop you using it for fun and profit.)

Is doctype.com a License Too Far for Stack Overflow?

Short answer:

No, because doctype.com doesn’t use technology licensed from Stack Overflow. Sorry. I got this one completely, completely wrong. D’oh.

Long answer:

This post was originally inspired by doctype.com. I now understand, thanks to an extremely informative comment from one of the doctype.com developers, that doctype.com doesn’t actually run on Stack Exchange. It looks and feels very similar, but is in fact a completely separate codebase built by the guys at doctype.com using Ruby on Rails.

This post is therefore based on completely incorrect assumptions. I’ve struck-out the bits that are actually factually incorrect, although my concerns about fragmenting the user based remain valid – even more so since I discovered that ask.sqlteam.com and ask.sqlservercentral.com are both Stack Exchange sites - but clearly doctype.com has nothing to do with it, and in fact, their platform offers a lot of design-centric tools that Stack Overflow doesn’t.

There’s also this disucussion at meta.stackoverflow.com that addresses a lot of the same concerns.

 

(Note: In this post, where I say Stack Overflow I’m referring to the website, and where I say StackOverflow LLC, I’m talking about the company  behind it.)

I’ve been using stackoverflow.com since it was in beta, and I love it. I ask questions. I answer questions. I hang out and read and comment and vote and generally find the whole thing a hugely rewarding experience. I think it works for two reasons.

First, the technology platform (now available as Stack Exchange – more on this in a moment) is innovative, usable and packed with great ideas.

Second, by actively engaging with people who followed Jeff Atwood and Joel Spolsky’s blogs, they gathered exactly the right audience to breathe life into their product. Stack Overflow launched with a committed, dedicated community of experts already in place. They created a forum where people like Jon Skeet will donate endless hours of their time for nothing more than kudos and badges. (I bet Jon’s employers are wishing they’d thought of that…)

Here’s a few choice quotes from Joel Spolsky’s Inc.com column I’m referring to (my emphasis)

“Between our two blogs, we felt we could generate the critical mass it would take to make the site work.”

“I started a business with the objective of building a big audience, which we would figure out how to monetize later.”

“we promised the audience that the site would always be free and open to the public, and that we would never add flashing punch-the-monkey ads or pop-up windows.”

Now this is the web, where “monetize” usually means “slap advertising all over everything.” – but when Stack Overflow introduced advertising, they were sympathetic and responsive to users’ feedback, and quickly evolved an advertising model that’s elegant, unobtrusive and complements the ethos of the site. The Woot! badge was clever. The tiny Adobe logo on tags like flex and actionscript was really clever – possibly the best use of targeted advertising I’ve seen.

Before long, non-programmers were asking how they could get a slice of the Stack Overflow goodness, and so serverfault.com – for systems admin questions – and superuser.com – for general IT enthusiasts – were born. That clearly worked, so they set up Stack Exchange, to license the platform to third parties, and soon there was moms4mom.com (questions about parenthood), Epic Advice (questions about World of Warcraft), Ask Recipe Labs (cooking and food), Math Overflow (for mathematicians), and various other Stack Exchange sites covering video, photography, car maintenance – all sorts.

A few days ago, I stumbled across doctype.com – a Stack Exchange site for HTML/CSS questions, web design and e-mail design – and some unsettling questions popped into my head.

1. Where am I supposed to ask my jQuery questions now?

I work on everything from T-SQL to a very occasional bit of Photoshop. There is a huge amount of crossover between HTML, CSS, Javascript, AJAX, and web server platforms and their various view/markup engines. Here’s the all-time most popular 20 tags on Stack Overflow, as of 20th October 2009:

1	c#	43,860
2	.net	24,590
3	java	22,924
4	asp.net	20,678
5	php	16,797
6	javascript	16,363
7	c++	15,462
8	python	11,639
9	jquery	11,287
10	sql	10,910
11	iphone	9,686
12	sql-server	9,165
13	html	7,932
14	mysql	7,794
15	asp.net-mvc	6,532
16	windows	6,425
17	wpf	6,370
18	ruby-on-rails	6,095
19	c	6,071
20	css	5,849

 

The highlighted rows are all Web client technologies – and that ignores all the questions that get tagged as PHP, ASP.NET or Ruby on Rails but actually turn out to involve HTML, CSS or jQuery once the experts have had a look at them. There’s clearly a thriving community of web designers and developers already signed up to Stack Overflow. Should we now all be asking CSS questions on doctype.com instead of using Stack Overflow? I have no idea! 

I realize there are HTML / CSS gurus out there who aren’t currently using Stack Overflow because they think it’s just for programmers – but wouldn’t it be better if Stack Overflow was looking at ways to attract that expertise, rather than renting them a walled garden of their own?  Getting designers and coders to communicate is hard enough at the best of times, and giving them their own “definitive” knowledge-exchange resources isn’t going to help.

2. What Does This Mean For The Stack Overflow Community?

Shortly after discovering doctype.com, I tweeted something daft about “stackoverflow failed as a business”, which elicited this response from one of the guys at Fog Creek… he’s absolutely right, of course. StackOverflow LLC is clearly doing just fine – their product is being enthusiastically received, and I’m thoroughly looking forward to their DevDays event later this month.

However, I think the success of StackOverflow LLC is potentially coming at a cost to stackoverflow.com – the site and the community that surrounds it – and in that respect, I believe that the single, definitive, free site that they originally launched last year has failed to fulfil its potential as a revenue stream.

The decision to license Stack Exchange to sites who are directly competing for mindshare with Stack Overflow’s “critical mass” worries me, because it suggests that StackOverflow LLC is now calling the shots instead of stackoverflow.com, and making decisions that are financially astute but potentially deleterious to the existing user base.

They are entitled to do this, of course. It’s their site, and I’m extremely grateful that I get to use it for free.

What’s ironic is that the worst case scenario here - for me, for stackoverflow.com, and for the developer community at large - is that doctype.com is wildly successful, becomes the de facto resource for HTML/CSS questions on the internet, generates a healthy revenue stream of its own, and StackOverflow LLC does quite nicely out of the deal. The format is copied by other technology sites, and soon there’s a site for SQL, a site for Java, a site for WinForms, a site for PHP… stackoverflow.com is no longer the definitive resource for programming questions, and we, the users, are back to using Google to trawl a dozen different forum sites looking for answers, and cross-posting our questions to half-a-dozen different sites in the hope that one of them might elicit a response. It’ll be just like 2006 all over again.

OK, So What Would I Have Done Instead?

doctype.com is trying to compete with an established market leader, by licensing that leader’s technology, in a market where the leader has a year’s head start and controls the technology platform. That’s like trying to open a BMW dealership in a town where there’s already a BMW factory outlet, run by two guys everyone knows and loves, whose reputation for service and maintenance is second to none. It has to fail… right? ^[1]

But – I can appreciate what they’re trying to do. I appreciate that StackOverflow LLC is not a charity, and I appreciate why the folks behind doctype.com think there’s a niche for an SO-style site focusing on designers.

The key to Stack Overflow’s success isn’t the catchy domain name, or that fetching orange branding. The key is the information and the people - I see no technical reason why something like doctype.com couldn’t be licensed as a front-end product that’s integrated with the same database and the same user community as Stack Overflow. Modify the back-end code so that users who sign up at doctype.com get certain filters applied. Use a different web address, a different design, maybe just include questions tagged with html, css, jquery and javascript to start with, so new users see content that’s more immediately relevant to their interests - but when they search or ask a question, they’re getting the full benefit of Stack Overflow’s huge community of loyal experts – not to mention the tens of thousands of accepted answers already in the Stack Overflow database.

How about it? doctype.stackoverflow.com, javaguru.stackoverflow.com, aspnetmvc.stackoverflow.com… each a finely-tuned filtered view onto a single, authoritative information resource for programming questions, from assembler up to CSS sprites. That has to be better than the gradual ghettoization and eventual fragmentation of a thriving community, yes?

Stop Press: Someone just pointed me at ask.sqlservercentral.com. That’s – yep, you guessed it – a Stack Exchange site for SQL questions. As if having to choose between stackoverflow.com and serverfault.com wasn’t bad enough. Does anyone else think this is getting a bit silly?

[1] Of course, it’s entirely possible that Joel & the gang know this, and are quite happy to take $129 a month off the folks at doctype.com whilst they work this out for themselves...