living with frameworks

Living With Frameworks

Stuart HerbertTechnical Managerwww.gradwell.com

[email protected]

1Welcome to my talk for the PHPUK 09 conference, presented at Olympia in London on Friday, 27th February 2009.

2Let me introduce myself. My name is Stuart Herbert. You have probably come across me through my blog, which is syndicated on Planet PHP.

3I’ve been programming since 1982, a professional software engineer since 1994, and I’ve been working almost exclusively on web-based projects since 1993. My career has given me the opportunity to gain a lot of real-world experience with UK household names over the years, and I’ve also been an active contributor to open-source software since the early 1990’s.

4I am currently the Technical Manager for Gradwell. We are what is fashionably called a Unified Communications company. We do premium quality broadband, email, hosting, and Voice-over-IP (VoIP). VoIP was the majority of our business in 2008; we are the UK’s third largest VoIP provider, and the current holders of the ITSPA #1 Best Business VoIP award. We have an annual turnover in excess of two million GBP. The company was founded, and is run by Peter Gradwell, who has been very active in the UK internet scene for the past decade.

5Today I’m going to share with you our experience of taking this (built without using any of the major PHP frameworks) ...

6... and turning it into this, which *is* built using one of the major PHP frameworks.

But I’m not going to talk about the code involved. If you’re interested in learning more about the technical details of your favourite framework, the ‘net is full of such information already. Instead, I want to talk about the wider context of what it can be like to bet the house on using a framework - what it’s like to try Living with Frameworks.


1. Why Use A Framework?

2.Adopting A Framework

3.Bringing New Developers On Board

4.The Costs Of Frameworks

7Because the idea of using a framework at all is still controversial in some circles, I’ll start by sharing the reasons why we decided to use a framework.






8I’ll show you the pain we’ve gone through when switching over to using a framework, in the hope that you’ll find things somewhat smoother :)






9As time has gone on, we’ve had to bring new developers into the company and into the team that builds our control panels, and I want to share their story and their experiences with you.






10And finally, I want to show you some of the pain points with frameworks, hoping that you can avoid these for yourselves when you adopt a framework.

Questions Welcome!(I’ll Be Asking Questions Of You :)

11I’m going to be asking you questions during the talk, so please feel free to ask any questions you have at any point during the talk. There will also be time at the end of the talk for questions.

12So let’s start by introducing you to my Rogues’ Gallery - some of the team at Gradwell who have been affected by the introduction of a framework.

Developers

Will Jonathan Tim

13Three developers who actively write coding using the framework are Will, Jonathan, and Tim. They joined Gradwell in the summer of 2008 as year-long placement students. They had no experience of programming with a framework before joining us.

Framework Guru

Dan

14We have a recognised role of Framework Guru, which I’ll explain in more detail later in the talk. Dan is our current resident framework guru, and has been using the framework for over two years now.

Old Lags

Ben Stuart

15Sitting outside the main development team, we have what I’m going to call a couple of old lags; old school PHP programmers who have been creating websites from long before Rails made frameworks fashionable. In our organisation, the old lags are also the most experienced engineers, but their day to day duties lie outside the control panel development team, which means that they really only get involved with code when time is of the essence.

In Gradwell, Ben and I both fall into the old lags category.

Management

Stuart Peter

16And finally, we have the pointy-haired bosses :)

Why Use A Framework?Living With Frameworks: #1

17I don’t think I can get away with *not* answering this question ...

Q: Already Using A Framework?

18Can I have a show of hands, please? Amongst the audience, how many of you are already using a framework? And how many of you are using a framework that you’ve built in-house?

(As you’ll see on the video of the talk, the majority of the audience at the talk were already using a framework)

4 Reasons Why We

Use A Framework

19There are many many reasons why you might be better off using a framework. We’re going to give you four that particularly resonated with us.

#1: Avoiding The Blank Canvas

20Reason #1: the scariest thing to a developer is starting a new project completely from scratch.

21Now, I promise - when I originally wrote this slide, I didn’t know I was going to scheduled against Microsoft out in the main auditorium!

This (and I don’t just mean an empty Notepad, although I think we can agree that Notepad *is* a scary sight :) is the blank canvas, and many developers find it the worst starting place of all. I’ve personally seen bright people, with a full Masters degree in computing, be completely unable to do anything when starting from here. But give them Ruby on Rails - a framework that comes with a process to follow to get started on a project - and they become extremely productive.

Why is that? To understand it, I need you to indulge me with a bit of a tangent ...

An Educational Model

22The reason why the blank canvas is such a challenge is down to how we mature in our understanding of any paradigm as we grow in experience.


Instructional

23When we start, we need to start at the Instructional level. A good example of this is a recipe. A recipe is a set of instructions, meant to be followed to the letter, in order to create a specific meal or food item. You’re not learning how to cook - at least, not directly - you’re learning how to cook something specific.

This is where new developers need to start. They need to be told what each step is that they need to do, and they need to be told how to do it, simply because they don’t yet know how to do it for themselves.


Instructional

Coaching

24After following instructions for long enough, the penny drops (for *some* people; sadly, not for everyone), and then it’s possible to move from an Instructional approach to a Coaching approach. Here, the developers are figuring things out for themselves, with management / senior staff using questions and directed experiences to help the developers improve bit by bit.


Instructional

Coaching

Collaboration

25Finally, once developers have learned how to self-improve, we reach working in collaboration, which is where we transcend being cogs in a machine and enjoy true teamwork together.

This model holds true for any form of education, be it scientific, creative, or sports, and it’s the model I successfully use to teach my Tai Chi students back home in South Wales.


Instructional

Coaching

Collaboration

?Developer

26Let’s take a new developer. The developer needs to start at the Instructional stage - they need to be told what to do and how to do it. A good framework (and this is something Ruby on Rails excels in) provides the skeleton to hang these instructions off. A framework is like the rules in a team sport in many ways.


Coaching

Collaboration

?Developer

27But what happens if there is no framework - there is no skeleton to base the instruction around?

When that happens, you have no choice but to move straight to teaching through coaching. This is where most managers jump straight to, and sadly has been made popular by otherwise well-meaning authors over the last decade or so.

But the problem is - you’ve skipped a step, and so have your developers. You have to instil the fundamentals first through instruction, before you can use coaching to help the developer to improve. Without the fundamentals, your developers are always going to have gaps in their understanding and ability.

Stuart

“Without a framework - your’s or someone else’s - how are you going to instruct your new developers?

What are you going to instruct your developers in?”

28Now, the framework doesn’t have to be one you download from the ‘net. It could be one you’ve built in-house, either deliberately, or one that has naturally evolved over time as you’ve re-used and improved your code time and time again.

Stuart

“Without a framework - your’s or someone else’s - how are you going to instruct your new developers?

What are you going to instruct your developers in?”

29But if you don’t have a framework, what are you going to instruct your developers in? Every job you give your developers is going to be new and completely different - but they’re not going to be ready for it.

#2: Code Outlasts People

30Reason #2: unless you work for a bubble start-up (not too many of those in the UK, thankfully), the code will be around longer than the individuals who work on it.

I’ll give you what admittedly is an extreme example. Back in the mid-90’s, I worked on an open-source project called NQS, the Network Queueing System originally written for NASA back in the 1980’s. One part of the code - the printer driver - was originally written in 1973. I was born in 1973, and I can see from the audience that most of you are a good deal younger than me :)

Because web development hasn’t been around for very long, I expect it’s difficult to imagine your code lasting anything like as long. But already standards and tastes are maturing, and you can take as an example something like phpMyAdmin which used to run under PHP 3 and now runs under PHP 5, without having been completely written from scratch along the way.

Code Outlasting People ...VoIP

Control Panel was written by ...

Ben31

Even in a company like Gradwell, where we have extremely low turnover, code outlasts people. True story. Our VoIP Control Panel was originally written by Ben, our Senior Developer.


Control Panel was maintained by ...

Ben32

He maintained it for a while after it was written ...


Control Panel is maintained by ...

Will Jonathan Tim33

... but he now spends his time on other things, so the VoIP Control Panel is now maintained by our three developers from the Rogues’ Gallery I introduced earlier. Now Ben hasn’t left the company; as we’ve grown, he’s moved on to other projects (as your senior people will do in your own firms as they grow).

But the code has outlasted the people.

Stuart

“Until the web came along, the vast majority of programmers were hired to maintain existing software - not to create new software from scratch.”

“This still holds true - even in the web development world - if you are hired to maintain a company’s own web applications, instead of working as a consultant / freelancer for other organisations.”

34I remember what it was like after I graduated in the mid 90’s. No-one I knew was hired to write new code, and until the web came along, none of us ever got to create new things. We were all hired to work on code that already existed.

It was the web that changed this, as we moved over to creating new websites for folks.

Stuart

“Until the web came along, the vast majority of programmers were hired to maintain existing software - not to create new software from scratch.”

“This still holds true - even in the web development world - if you are hired to maintain a company’s own web applications, instead of working as a consultant / freelancer for other organisations.”

35If you work on in-house web-based apps, such as products or internal systems, then this still holds true.

Developers simply don’t get a lot of practice creating new non-trivial apps from scratch, which re-enforces the problems behind reason #1 too.

#3: Embrace Best Practices

36When I asked Dan why he got out of using a framework, he was quick to point out how a framework makes you raise your game, because the right framework can introduce better ideas and practices to the way you do things.

Or, to put it another way ...

37

Today, we’re happy with what we produce ....

but tomorrow we want to do better.

38One of the things I always aim for in my staff is that drive to improve, to get better and better at what we do. We want to achieve better results, and we want to do so in better and better ways as time goes on.

Dan

“Good frameworks embody best practice use of design patterns.”

“A framework encourages code reuse.”

39Dan strongly feels that good frameworks bring with them best practices on how you design your code, so that you can develop quickly now, and have less maintenance in the future.

Dan

“Good frameworks embody best practice use of design patterns.”

“A framework encourages code reuse.”

40And a key part of this is also that by using a framework - code that’s meant to be re-used, perhaps even code you’ve downloaded in order to re-use - it encourages you to make your own code re-usable.

And we’ve certainly seen the benefits of this with the Twittex project that I talked about at PHP NW in Manchester in November 2008.

Stuart

“Frameworks bring a common discipline that can be imposed across the entire development team.”

“Incorporating a framework into our development practices helps us in our drive for continuous improvement.”

41From my own perspective, I’m a big fan of the common discipline that a framework brings. Discipline is a big part of building any successful team, and adopting a framework allows you to go much further than you can simply by imposing coding standards on a team.

Stuart

“Frameworks bring a common discipline that can be imposed across the entire development team.”

“Incorporating a framework into our development practices helps us in our drive for continuous improvement.”

42But you have to remember that a framework is only part of your wider development standards (which, I guess, is the whole point of this talk). It isn’t a silver bullet, but it’s certainly an important part if you want to drive for continuous improvement.

#4: Frameworks Save Time

43Our fourth and final reason: frameworks save time.

Once you’ve mastered a framework, they help you get more done in less time. They make individual developers and teams alike more productive. We’re in a recession atm; possibly even a depression. If it isn’t already for you, time *will* become a competitive advantage.

Q: Has using a framework saved you time?

44A show of hands please - how many of you have found that you save time if you use a framework?

(A majority of people agreed with this, but by no means everyone).

Tim

“Development is quicker, but only when you do things the way the framework wants it done.”

“Every time we deviate from the framework, it always comes back to be a pain, and has to be refactored.”

45When I was planning this talk, I asked my developers what they thought about frameworks. Tim in particular had a couple of important points to share on this. He reminded me that, in his experience, it is quicker to develop using a framework, but that it’s essential to embrace the way the framework wants things to be done.

Tim

“Development is quicker, but only when you do things the way the framework wants it done.”

“Every time we deviate from the framework, it always comes back to be a pain, and has to be refactored.”

46Every single time we’ve ignored the framework, where we’ve thought we could be cleverer, it has come back to bite us on the backside, and progress has been halted until the code has been refactored.

That’s a important lesson we’ve learned that I urge you to take away to share with your colleagues and teams back at your companies. A true framework (as opposed to a component library) imposes a discipline that comes from a specific way of thinking. To get the benefits of quicker working using the framework, you have to embrace that way of thinking.

At Gradwell, we’ve gone one step further, and given someone the job of policing this too, which I’ll come on to in a bit.

Dan

“New versions of the framework bring with them new features that you don’t have to write yourself.”

“There is help and support through the framework’s developers and wider community.”

47From Dan’s perspective, adopting someone else’s framework also brings some useful things that save a lot of time and trouble. New releases of the framework will inevitably bring new features that you can use without having to write them yourself.

Dan

“New versions of the framework bring with them new features that you don’t have to write yourself.”

“There is help and support through the framework’s developers and wider community.”

48And - something that we’ll come back to in a moment - good frameworks are backed by its developers and the wider user community, which are both good places to get help and support from.

Stuart

“Frameworks allow you to solve your common problems once, so that the majority of your time goes on solving the problems unique to your business.”

“We’re now in a global recession - perhaps even a depression. If your competitors get there first, will there be enough business left over for you?”

49As a manager, looking after products that must evolve year on year, I love the way that frameworks allow my developers to solve common problems just once (benefits of code reuse). As the guys get better at using the framework, so they end up spending more and more time solving the problems that are unique to our business, which means we can get more and more done.

Stuart

“Frameworks allow you to solve your common problems once, so that the majority of your time goes on solving the problems unique to your business.”

“We’re now in a global recession - perhaps even a depression. If your competitors get there first, will there be enough business left over for you?”

50A quick show of hands: how many of you worked during the last recession? (Not many)

It’s widely accepted now that we’re in an economic recession. This is new ground for most of us, an environment most of us have no working experience of. And we haven’t hit the bottom yet - this might even turn into a depression, which will be something none of us have experienced before.

In these times, with firms going under daily, there are less and less customers for us to chase. (There are also more and more people chasing these customers, as laid-off folks go freelance to try and find a new income). How long it takes you to deliver a solution - time - will become more and more of a competitive advantage as the recession continues. If your competitors can deliver a solution to a customer’s needs in less time than you can, which company do you think they will pick? And, as the economy gets worse, will there be enough business left over for you?

Q: Why Do You Use A Framework?

51A question for those in the audience who have already adopted a framework: what made you decide to do so? Any of the reasons I’ve already mentioned, or was there a different reason that made it seem like a good idea at the time?

Q: Why Don’t You Use A Framework?

52And for everyone else - what are the reasons why you don’t use a framework yet?

(Aral made a great point here: he doesn’t use a framework because he feels that it stifles the creative process. This would be a great topic to explore as a talk in its own right in future).

Adopting A FrameworkLiving With Frameworks: #2

53So those are the reasons why we decided to use a framework, and together we’ve shared some of your reasons for and against too.

Next, I’d like to look at what it is like to actually make the switch to using a framework. I hope that there are some useful lessons to learn from our own experience of having done this.

What’s important to you?

54Let’s start by actually picking a framework. The key question here is to understand what it is, from a framework, that is truly important to you and your particular needs.

How We Picked A Framework

We tried building one in-house, but we didn’t like the results

Dan looked outside for something better

Features mattered most

But community mattered almost as much

55For ourselves, well - we’d already tried building our own in-house. It’s still in use today, but we didn’t like the results, and it will be phased out soon.






56So the decision was made to go and look at what frameworks existed at the time.

57Choosing a framework today is a difficult decision. There are hundreds to choose from, and millions of web pages and blog posts with their own advice on the subject!






58The features we needed from the framework were the most important thing we considered when choosing a framework.

59Figuring out the features provided by a framework can be a challenge. I wasn’t surprised to find that symfony’s website didn’t publish a clear features list ...

60... but I was surprised to find that Zend didn’t publish any such list on a page that I could just point at for this talk. If you want your framework to stand out from the crowd today, this might be something worth incorporating into your framework’s website.






61A vibrant, active, and supportive community was almost as important to us as the framework’s features. There’s nothing worse than downloading a large bunch of code and being effectively on your own with it.

62I don’t mean to pick on anyone here, just to share our experiences with the frameworks we looked at at the time. One of the frameworks we evaluated was Zoop, but as you can see here it’s community isn’t very active.

63We also looked at CakePHP. At the time, we found CakePHP to also have a quiet community, but looking at it today, it’s clear that CakePHP has a very active community.

64Ignoring the rendering problems with symfony’s website :) we can see that Symfony’s community is extremely busy today.

Framework Guru

Dan

65Something that has naturally evolved during our adoption of a framework is the role of a framework guru. This is a role that you might not have in your own company atm.

So what does a framework guru do, and why have we ended up needing one?

What A Guru Does

Tim

Jonathan

WillDan

sets standards

66The first thing Dan does is set development standards for the rest of the developers to follow. This is very similar to the coding standards traditionally set by senior developers, but has a wider scope to include unit test practices and our ontinuous integration approach.

What A Guru Does

Tim

Jonathan

WillDan

reviews code

67Another thing Dan does is review the code produced by the developers. Earlier in the talk, I mentioned that we’ve found it’s very important to do things the way the framework’s way. One of the reasons for Dan leading the reviews is that it allows him to ensure that we’re doing just that.

What A Guru Does

Tim

Jonathan

WillDan

refactors code

68When design problems are found, the sooner they are corrected the better, especially if they’re down to not using the framework the right way. When these problems are discovered, Dan leads the efforts to refactor the code.

(As an aside, if you ever find yourself working for someone who actively fights against code refactoring, run away as quickly as you can. The ongoing costs and pain from not refactoring are horrendous).

Stuart

What A Guru Does

Dan

researches

Peter

the future

69Dan also keeps one eye on the future. As new versions of the framework are released upstream, Dan masters them and figures out how we can migrate all of our code as painlessly as possible over to the new version.

Stuart

What A Guru Does

Dan

drives

Peter

improved practices

70And finally, he provides a focal point for the management team to drive improved practices into the wider development team. He provides a deep insight into our experiences so far, which always proves very useful in looking at the likely impact of proposed improvements to our practices.

Framework Guru Role

Someone on the payroll, not someone from outside

The champion for the framework

One eye on the future

Part of the wider development practices context

71In the past, I’ve been a contractor brought into a team to deliver improvements, and it’s something I’ve seen other teams / departments do in other firms time and time again. I’m not saying it can’t be made to work, but in general I’ve seen far more success occur if the person in this role is a permanent member of staff. There’s nothing wrong with them bringing in outside help to build their own skills and understanding, but if you don’t have the permanent member of staff, then when the contractor / consultant leaves, invariably so does the drive to continue to do this work.

This role is something you need every day, not a quick fix every now and then.

Framework Guru Role





72The framework really needs a champion, someone who has a deep understanding of how the framework needs to be used and who has a great passion for ensuring that it is used that way by everyone else. The framework can’t speak for itself. It needs a champion.

Framework Guru Role





73As mentioned before, he keeps one eye on the future, to do his best to ensure that we’re not heading up any obvious blind alleys.

Framework Guru Role





74... and, in Gradwell, he also carries out traditional Senior Developer role duties, because a framework is only part of the wider discipline of software development.

The Challenge With Legacy Code

Legacy code has to be replaced quickly

Otherwise, the goal posts move

Problem is, the legacy code won’t refactor easily to fit the framework

Avoid maintaining both the legacy and the new framework code - very expensive!

75I want to talk about legacy code now, because adopting a framework will invariably mean treating most (if not all) of your existing code as legacy code. What is legacy code? It is code that should no longer be maintained, and that should be phased out of use.

The first thing to say about legacy code is that, once you start replacing it, you have to complete the job as quickly as possible.






76Why? Well, until the new code is ready, the legacy code is still there in production, still in use. The longer the legacy code is still in use, the more pressure will come from the business to fix more bugs, and add more features. As weeks turn into months, this will mean that the goal posts will have moved - the new code will need adapting to keep up with the changes in the legacy code. This can be an error-prone process (too easy to forget to bring an important bug fix or small feature change over to the new code), and it’s also very demoralising for developers the longer it goes on.






77That speaks for itself, really. Because a framework imposes a different discipline to the one you already program with (assuming you’re not replacing spaghetti code), chances are that your legacy code can’t be easily brought across and made to sit inside the framework. If your experience is anything like ours, the code needs replacing, not refactoring.

And it’s not just the code that won’t refactor. You’ll probably find that management are looking at this as an opportunity to change the product too. I know that I’m guilty of this from time to time!






78I can’t emphasise this point strongly enough. Maintaining two sets of code means all changes have to be done twice - once in the legacy code, and once in the new framework-based code. It’s very expensive, and not just for your development team. Mistakes in this process also generate extra support tickets and calls, and can lead to customers being given discounts in order to keep their good will.

A True Story Of Legacy Code

Written before a framework adopted - legacy code

A plan put in place to rewrite it using the framework

It was re-written, but the plan had focused on the wrong features

Still in production today

VoIP Control Panel

79Remember the VoIP Control Panel I introduced earlier in the talk? I’d like to share with you a true story of our own attempts to replace the legacy version of this with framework code.

A True Story Of Legacy Code





VoIP Control Panel

80In order to replace it, a plan was put together, and a developer was assigned full-time to move it across to the framework.

A True StoryOf Legacy Code





VoIP Control Panel

81This he did, exactly as requested. What he delivered worked, but there was no way I could agree to putting the code live. I’m passionate about the need to avoid maintaining both legacy and framework code at the same time, but unfortunately when the plan was devised, it didn’t include all of the features required to replace the legacy code. Quite understandably, the focus had been on proving that we could implement a working control panel using the new framework, but this had been at the expense of putting out something complete enough to replace a discrete chunk of legacy code.

A True StoryOf Legacy Code





VoIP Control Panel

82As a result, the legacy VoIP Control Panel continues in production today, until the remaining features (white label / co-branded partner support) have been added to the new code.

Stuart

“Legacy code is like a vampire - kill it properly or it will come back time and time again to bleed you dry.”

“Beware of anyone who manages code like an ostrich - they create legacy code vampires.”

83The point I want to really drive home - pun intended - is that legacy code is a vampire that needs to be well and truly staked, otherwise you’re not going to kill it, and it’s going to come back and try to suck all the blood out of you.

Stuart

“Legacy code is like a vampire - kill it properly or it will come back time and time again to bleed you dry.”

“Beware of anyone who manages code like an ostrich - they create legacy code vampires.”

84And, from places I worked before Gradwell, I want to add this cautionary note. Legacy code is a real problem, and to inexperienced (and non-technical) managers, it can look like a problem too big to do anything about. This is a really dangerous attitude to legacy code, because these managers end up creating vampire after vampire of legacy code which really add up to cause major costs and moral problems.

Frameworks Don’t Fix Broken Practices

Switching to a framework stresses your development practices more than usual

Weaknesses in your practices (specifications, quality) will be laid bare

Fix them using people, not technology

85Looking back at the story of what happened when trying to replace the legacy control panel, you might be sat there wondering how come the mistakes were made. From our experience, day to day development practices were geared towards maintenance of existing code - fixing bugs and adding new features. The decision to start again, building new code around a recognised framework, meant shifting over to a different kind of development - and, as a result, this stressed our development practices in new ways.





86Because our development practices were put under stress, the weaknesses we had in our practices at the time became magnified. And it was these weaknesses that directly lead to our failure to replace the legacy code at the first attempt.





87But how do you go about fixing these weaknesses when they become apparent? The best way is to always fix them using people rather than technology. Technology provides tools, but its people who do the work, and ultimately when adapting development practices, its the work done by the people that needs to evolve.

Bringing New Developers On Board

Living With Frameworks: #3

88Over time, the makeup of your team will change. As the company grows, people will move on - some to other roles in the organisation (as Ben has done for us), and inevitably some will leave. So you need to know how to bring new developers into the team, and how to make them productive using whatever framework you choose.

Can you hire developers already skilled in your framework?

89Here’s a question for the audience who have already chosen a framework: are you able to hire developers who already have experience using the framework you’ve chosen?

We find it difficult.

90So far, we’ve found this difficult.

So we hire developers anyway, and train them up.

91Rather than focus on hiring the most experienced framework developers we can find, instead we’ve focused on hiring the best developers we can, and then training them up in the framework of our choice.

Developers

Will Jonathan Tim

92Will, Jonathan, and Tim are all developers we have hired since adopting a framework. None of them had any experience with our framework of choice before joining the company.

For those of us who have been working as programmers for a long time, it can be difficult to put ourselves into their shoes, to understand what it’s like to be a young programmer coming into a new company with a new approach to learn. So I interviewed Will, Jonathan and Tim, and decided to present their answers in their own words for this section of the talk.

Did you do any research on the framework before joining the company?

Will Jonathan Tim

93The first thing I wanted to know was to understand how prepared they were before joining the company. What did they find out about the framework we use?

“Read some of the manual, did a bit of coding but didn’t really grasp how the framework was meant to be used.”

Will

Jonathan

Tim

“Emailed existing staff, read the manual, but didn’t do any coding before joining.”

“I looked at it before joining, found it a bit different, and gave up on it in the end.”

94Will made a particularly insightful response. Looking back now, he realises that his research at the time didn’t help him fully understand how the framework was meant to be used. His research didn’t help him truly learn how to think in terms of how the framework needs to be used.


Will

Jonathan

Tim



95Jonathan was a little more pro-active on the experience front, emailing our other developers to learn more about the framework. However, he didn’t follow that up with any hands-on coding.


Will

Jonathan

Tim



96Tim’s research ultimately hit a dead end. He found the framework to be a different approach to what he was used to.

So the main point I’ve taken from these answers is that there are important limits on how prepared new starters are likely to be when starting a new job to work with a framework.

Did using a framework worry you before joining the company?

Will Jonathan Tim

97I was curious to find out why these limits existed, and in particular whether or not the framework itself was ultimately the problem.

“Probably would have done if I had know more about frameworks, because it’s very different programming with a framework.”

Will

Jonathan

Tim

“I would have been worried if it had been a proprietary framework.”

“A bit of trepidation, but figured I would be using it day to day and would have no trouble picking it up.”

98Will wasn’t worried at the time, but looking back now, he feels that he would have done if he’d realised just how different it is to work with a framework.


Will

Jonathan

Tim



99Jonathan’s answer surprised me. Because we use an open source framework, one you can freely download from the ‘net, he wasn’t worried. But he would have been worried if it had been a proprietary framework.

(The audience at PHP UK 2009 speculated that this might have been because a proprietary framework isn’t a useful thing to have on a CV).


Will

Jonathan

Tim



100Tim was a bit worried, but he assumed that he’d have no trouble learning it on the job because he’d be using it day in and day out.

Stuart

“Your staff expect to be trained when they join a new job.”

“Frameworks do not negate the need to train your people.”

“Training is an opportunity to build a team the way you want it to work - not a burden to be avoided.”

101So what are the key points I’ve taken from this? The first one is that your staff have an expectation that they will be trained. I partially blame UK GOV and the last twelve years of namby pamby education that they’ve presided over, but it’s also because this is no longer the pioneering industry us older folks originally joined.

Stuart




102Picking a framework doesn’t mean that you can get away with not training your people. If anything, the opposite is true, because as we’ve repeatedly seen during this talk so far, a great deal of the success comes from being able to think in the way that the framework wants.

Stuart




103Too many firms view training as an overhead, an activity that adds no value at all. I agree that sending folks off to day or week-long courses is often of limited use (I prefer to do as much training as possible in-house), but I want to encourage you all to see how the right training can be used as a great way to build a real team.

Inducting New Developers

104So how do we go about doing just that? How do we bring new developers into the team?

What Is The Goal?

1. To make new developers as productive as possible as quickly as possible

2.For new developers to get the difference between using a framework and not using one

3.To give new developers a sense of confidence

105I think it will help if we first look at what we’re trying to achieve with training new staff. The most important goal of the training is to get new developers up to an acceptable level of productivity as quickly as possible. Your developers cost you money, and it is money you have to find every month in order to pay their salaries. The sooner their activities pay for their salaries, the better.

What Is The Goal?




106We’ve seen the recurring theme that using a framework is about a change of thinking first and foremost, not a change of programming. Therefore, it’s important that the new developers are made to think about what this change might be - preferably, they need to be able to experience it for themselves.

What Is The Goal?




107Finally, we want the new developers to complete their induction with a sense of confidence that they can take into their first real project. They will feel better, which is always important, and they will be more motivated to tackle the steep learning curve that will still be before them.

The Project

108When our new developers join, we give them a project to perform. We give each developer the same project, and we have no intention of ever using the code in production. It’s a nice, safe sandbox for the developers to learn more about how we do things, and for us to learn more about how the developers do things.

The Project

Web Hosting Control Panel

109I’m a great believer that the project should be relevant to whatever your firm actually does. One of the things we do is web hosting, so our induction project is based around the requirements for a brand new web hosting control panel.

The Project


MySQL Database

110We also ask the developers to develop the database schema from scratch. In my experience, this area is a weakness in most web developers these days, so I’m keen to give our new developers every opportunity to improve their skills in this area.

The Project


MySQL Database

Apache + vhosts

111I’m also a great believer that code doesn’t exist in isolation, and that web developers should understand how Apache works and be comfortable setting up basic websites using Apache. As the web hosting control panel will be managing virtual hosts on Apache, I ask the developers to setup Apache and some vhosts as part of the project.

The Project


MySQL Database

Apache + vhosts FTP Uploads

112And finally, we ask the developers to also setup an FTP site for customers to be able to upload their files to the virtual hosts managed through the control panel. Personally, I abhor the idea of using FTP for this sort of thing, but sadly the wider world (and their web development tools) still hasn’t adopted an acceptable secure standard such as SSH/SCP, so hosting firms like us are stuck with having to provide FTP access for the forseeable future.

Once complete, the end result is a little standalone control panel solution for web hosting, one that’s complete enough to actually use and play with. Although it’s built around the framework, it’s also focused on delivering a customer-centric solution, which makes it realistic, and helps prepare new developers for working in real projects.

First time, whatever way they’re used to programming

Second time, make them use the framework

Do The Project Twice

113Now here’s the key to it all. Have them do the project twice. First time around, let them write the code in whatever style or approach they currently prefer. This means that they stay within their comfort zone, which plays a great part in them getting started working for you.

It also means that you get to see first hand what they already know, and a little bit about their subconscious preferences.

First time, whatever way they’re used to programming

Second time, make them use the framework

Do The Project Twice

114Then, repeat the exact same project, but this time the PHP code has to be written using the framework.

Because they solved the problems around the customers’ requirements first time, they’re still in a comfort zone. They now know what they have to deliver, which leaves them free to focus more on how to deliver it using the framework instead of their usual coding approach.

What did you think about the induction process?

Will Jonathan Tim

115I asked Will, Jonathan, and Tim about their experiences of having done this induction process for themselves.

“Thought it was good.”

Will

Jonathan

Tim

“Doing the project twice really showed the difference the framework made.”

“Very good - it allowed me to compare previous approach to that of using a framework.”

116Will kept it nice and simple. He thought the induction process was good.


Will

Jonathan

Tim



117Jonathan picked up on one of the key goals behind the induction. Running the project twice - getting him to compare and contrast his approach to the one the framework requires - made him see the difference that the framework makes.


Will

Jonathan

Tim



118Tim also made the same point. The induction process got them all thinking.

How would you improve the induction process?

Will Jonathan Tim

119But I’m always keen to improve things for the future, so I asked them for their ideas about what we could change for our next intake of developers.

At this point I need to say that, for commercial reasons, immediately after they had completed the induction, I had to put the guys onto maintaining our legacy VoIP control panel (legacy code vampire strikes again!)

“It would have been better if we had used the framework in anger immediately after induction.”

Will

Jonathan

Tim

“The induction didn’t make enough usage of the bits of the framework we’ve extended.”

“I didn’t use the framework after the induction, so what I learned faded a bit.”

120Will made the excellent point that not being able to use the framework immediately after the induction made the induction less useful than it was intended to be.


Will

Jonathan

Tim



121Jonathan made a different point, but a very valid one. If you’ve lived with a framework for awhile, then you will inevitably end up extending it and adding your own components too, components specific to your business. We expect our developers to use these Gradwell-specific components in their day to day work, but we didn’t include these components in our induction process at all. That is something we need to change for next time.


Will

Jonathan

Tim



122Tim also made the point that what he learned in the induction faded a bit, because he didn’t get to use it immediately after the induction.

Stuart

“You can’t beat hands-on experience as a teaching method.”

“Repeating the work allows developers to compare and contrast - it makes them think.”

“Use it or lose it is an important principle that always holds true.”

123So what are the points I’ve taken from this? First of all, hands-on experience is a great way to teach new concepts to people. When people are taught something, they understand it intellectually, but when they experience it for themselves, they know it.

Stuart




124The induction works because of the opportunity it provides our developers to compare and contrast their old way of doing things with the way that the framework demands.

Stuart




125And that old chestnut - use it or lose it - applies just as much to learning about frameworks as it does to learning anything else. Follow up the induction with real work (I recommend at least 3 months) of using the framework in anger. It’ll really help what they’ve learned to stick, and for their learning to accelerate.

But What AboutThe Old Lags?

Ben Stuart

126I’ve talked about bringing new developers on board, but what about the old lags - the old developers you have elsewhere in the company?

What The Old Lags Say ...

Our main purpose isn’t new web development

No time to gain experience using chosen framework

This is a good thing, because it stops us getting dragged into new web development unnecessarily

Can only offer limited help in emergencies

127The old lags tend to be developers who have moved on to more senior roles as the company has expanded. Both Ben and I have done that (Ben now divides his time between our VoIP platform and our billing system, and I was brought into the company as dedicated technical management), so although we’re both highly experienced developers, neither of us sit down and write web code on a day to day basis.






128As a result, it’s difficult for us to find the time to get down and dirty with the framework.






129Privately, we both admit that this is a good thing. We can’t get dragged away from our own jobs to help out with new development.






130But it does cause problems at those rare times when emergencies happen. In an emergency, every minute counts. We’re both totally comfortable with taking legacy code apart in moments to diagnose and fix an urgent bug, but frameworks typically come with things that are somewhat alien to PHP, such as application build steps, and these differences make it much harder for the old lags to just dip into the code for five minutes when customers are screaming down the phone.

The Costs Of Frameworks

Living With Frameworks: #4

131So far, I’ve spoken about the benefits that frameworks can bring, how we went about adopting a framework, and how we go about bringing new developers into the team. But there are downsides too, and I want to finish my talk by quickly looking at a few of those.

The Learning CurveIs Steep

132Learning how to use a new framework takes time - a lot of time. This is because you have to change how you think about the code you’re going to write. And it’s also because some frameworks introduce additional steps and tools that you would never need in more traditional PHP programming practices.

Will

“You can’t always see how the code needs to be structured up front.”

“You can easily design something with an assumption that proves wrong when you get into the coding.”

“These problems always lie in the difference between the model, view, and controller.”

133Will had some great points to make about this. The first thing he points out that, when you’re learning a framework, it can be very difficult to see what the code structure should be at first.

Will




134This makes it very easy to design a structure up front that doesn’t work when you try to work with the framework.

Will




135The structural problems, in the framework we use, are always down to the differences between the model, the view, and the controller.

Frameworks Hide StuffEspecially Database

Queries

136Now this is a bug-bear of mine, I freely admit :)

Stuart

“Frameworks are designed to abstract away things developers routinely struggle with.”

“Many developers find SQL hard, hence the rise of the ORM.”

“The number - and performance - of database queries is my number one issue managing projects using frameworks.”

137One of the advantages of a framework is that they hide away some of the difficult aspects of creating web based applications.

Stuart




138SQL in particular is something that many developers find very difficult, which has led to the proliferation of Object-Relational Mappers (ORMs) based around the ActiveRecord pattern.

Stuart




139But, because developers don’t understand SQL, they don’t understand what the ORM does either, and current ORMs are too immature to help developers out when it comes to performance. As a result, it’s far too easy to end up with a web page that makes far too many database queries, and far too easy to end up with database schemas that are not optimal for performance. I’m a bit old-fashioned here; any web page that makes more than ten database queries is making too many queries for my liking. But, thanks to the use of ORMs, I’ve seen pages that make hundreds of database queries, and I’ve also seen ORMs that create very badly performing SQL queries.

Let me be clear here. I think ORMs are a good idea, when the developer using them has a thorough understanding of what they do and the costs involved. The problem for me is that they’ve become a crutch to too many developers who simply don’t understand what’s going on underneath. Badly used ORMs consistently cause problems and delays with projects I manage, and as a result they’re the number one problem that frameworks cause me.

You Can’t Just Hack At Stuff Any More

140This is something I’ve touched on a couple of times earlier in the talk. Each framework is designed to be used in a specific way. If you don’t respect that way, you get your fingers burned in the end.

As a result, you just can’t take a flyer at projects any more.

Stuart

“Adopting a framework for us was part of an overall move from one-man projects to true teamwork.”

“If you don’t do things right when using a framework, you have to go back and rewrite them sooner rather than later.”

“But is this a bad thing? Only if the organisation does not mature around you.”

141So, the positive: frameworks are, in our experience, a great help in moving away from one-man projects into real teamwork, because they impose a common discipline that has to be respected.

Stuart




142If you don’t respect the framework, you either end up with horrendous costs, or you go back and refactor stuff as early as possible. This constant need to respect the framework slows you down for a while, until you learn to do this stuff in your sleep.

Stuart




143But is it really a bad thing? If you do things right, then across developers of all ability, you’re much more likely to deliver code that works first time and that you don’t have to constantly go back and firefight. So it can actually be a good thing, but only if the rest of your business is maturing too, and recognises that you need to shift from “take a flyer -> firefight” to “deliver -> next project”.

Don’t Forget The Need To Upgrade The Framework!

144Finally, I want to point out something that I’ve seen overlooked time and time again : the need to be able to upgrade the framework affordably.

Framework Upgrades

You need upgrades, if only for security fixes.

Prepare for an upgrade by constantly staying on top of quality and best practices.

Assume a framework upgrade will break backwards compatibility.

Get it wrong, and the costs are horrendous.

145Why do you need to upgrade? I’ve already talked about getting new features, but the main reason you need to be able to upgrade is to be able to pick up security fixes.

Framework Upgrades





146In our experience, upgrades are much easier if you’re always making sure that your code is sticking as closely as possible to the framework’s way of doing things. The more weird stuff you try to pile on top, the harder your life is going to be when you decide to upgrade.

Framework Upgrades





147That said, you still have to assume that a framework upgrade will break backwards compatibility. You need to build an appropriate amount of time into your plans and budgets to upgrade your code and thoroughly test it.

Framework Upgrades





148Because, if you don’t, the costs can be awful. One place I’ve worked only allowed a few days for upgrades that actually took weeks to achieve. Someone has to pay for all that time, and if the difference between the budget and reality is huge, then your clients may not be happy to pick up the extra costs, and it might just end up being your company that has to pick up the bill.

To Recap ...

149So that’s what it can be like to live with a framework for years at a time. To recap all of the points I’ve made in this talk ...

Why We Use A Framework

1. Avoid blank canvases

2.Code outlasts people

3.Embrace best practices

4.Frameworks save time

150






151






152






153

Adopting A Framework

1. Work out what matters most to you

2.Hire someone to be your framework guru

3.Have a plan to deal with legacy code

4.Frameworks don’t fix broken development practices

154






155






156






157


1. Can you hire developers who already use your framework of choice?

2.A structured induction process really helps

3.Follow induction up with several months of use of the framework

4.Don’t forget other developers outside the team

158






159






160






161


1. Steep learning curve

2.Hidden processing, esp database

3.Perception of slow progress

4.Framework upgrades

162






163






164






165

Our Final Thoughts

166I want to end this talk by sharing a few final thoughts from everyone with you.

“Everyone should learn to use at least one. They are a good way to learn how design patterns should be used.”

Will

Jonathan

Tim

“I wouldn’t consider doing a decent sized PHP project without using a framework.”

“They can be a bit bloated, but very nice to work within as a development tool.”

167I asked our developers what they thought about frameworks now, and whether or not they’d do their next personal project with a framework. Will feels that experience with a framework is very important.


Will

Jonathan

Tim



168Jonathan went further, and felt that he wouldn’t consider doing a decent sized PHP project without using a framework.


Will

Jonathan

Tim



169For Tim, it was more a case that frameworks are nice, but they’re not perfect.

Dan

“Adopting the framework has accelerated our drive towards best practices and has been both educational and productive.”

“Refactor early, refactor often.”

170Dan was very positive about frameworks, and felt that adopting a framework has played a large part in improving our overall development processes.

Dan

“Adopting the framework has accelerated our drive towards best practices and has been both educational and productive.”

“Refactor early, refactor often.”

171This isn’t an actual quote from Dan, but if you asked anyone else in the team what Dan would say about working with a framework, this is what they’d say he would say :)

Stuart

“Any piece of code that survives long enough either becomes a framework, or needs to use someone else’s.”

“The framework must fit into your overall development plan and practices.”

172For myself, time and time again I’ve seen code that lasts long term go two ways. It either effectively becomes a framework - it becomes standardised and more and more structured - or it turns into spaghetti code that would greatly benefit from being replaced with a recognisable framework.

Stuart

“Any piece of code that survives long enough either becomes a framework, or needs to use someone else’s.”

“The framework must fit into your overall development plan and practices.”

173Frameworks are a key part of software development, but they are no silver bullet. They have to be part of an overall plan and an overall set of practices.

“I think the apps we've produced using a framework are better than the ones we produced before and I am dead keen on the whole framework + testing regime.”

“I am generally frustrated that they contain so many bugs and don't work first time round - and at the (lack of) pace of development - but I don't put that down to the adoption of a framework.”

“If the extra time means it is done right first time, then that's an okay trade off, as long as we don't end up missing the boat entirely.”

Peter

174To finish, I asked Peter Gradwell, our company owner and Managing Director, how he felt about our move to frameworks. He has been incredibly supportive of the move to frameworks so far.




Peter

175He is currently concerned about both quality and the time it takes to deliver new code, but interestingly he doesn’t feel that that has been caused by the move to using a framework.




Peter

176He also brings up the importance of time to market. Sometimes, there are commercial deadlines which must be met, and development processes and practices have to be flexible enough to accommodate that, otherwise the business itself may run into trouble.

Your Thoughts?

177That’s a summary of our experience of living with frameworks. It’s important to remember that this is just our experience - both Gradwell’s over the last couple of years, and my own over a career spanning back to before the web and before PHP.

Thank you for viewing this presentation. I hope you’ve found it interesting and that you’ve learned something from it. We certainly learned a lot from doing this project. If you have any questions or feedback, please get in touch with me [email protected].