3:50 pm dropbox case study

• How we build things?
  • We start with great people.
  • Hiring fewer but better people reduces need for coordination. Less things to share among less people.
  • Using etherpad to plan design with trac
    • Taken point, you don’t need an incredible infrastructure to coordinate. Simpler is better.
  • small loosely-couple teams formed from a 25-person engineering team.
    • each team is locally organized
• Find great people and get out of the way
  • no mandatory hours
  • make the office a nice place to work at
  • very heterogeneous environment when it comes to people preferences to workstation.
• Decentralize day-to day decisions through culture & values
• Co-founders maintains the soul of the product & user experience
  • Designer is the keeper of the look & feel
• Big results with small # of people
  • one visual designer (formerly community manager)
    • got lucky. He had no prior experience in design
  • server team of 3 manages 100+ billion files …
  • Strategy: divide & conquer, keep teams small
  • as team grows, you need to institutionalize certain things like values, culture & mission
• Planning
  • we don’t do a lot of long term planning
  • But we need more than we used to
    • things get more complicated over time
    • less information spreads by osmosis
• Dropbox company goals
  • modeled after Google’s OKR system
  • yearly goals & quarterly goals
  • forms a hierarchy that is shared publicly
    • overall company, product, team, individual goals
  • Perfect is the enemy of good enough
    • your planning needs iterations too
• Study how other companies grew
  • Challenge with scaling orgs is what used to work start failing quietly
    • new engineer doesn’t really know how their stuff fits that big picture
    • where the company is going
    • creates malaise that flares up
  • Try to learn from other companies growing pains
  • Take comfort in that all other company had problems too.
    • We talked to our investors who invested in other companies for sanity check and discovered that every company is special in its own special way
• Challenge we faced at scale
  • launch fast and iterate quickly
  • different sub-teams need different engineering tradeoffs
  • our most valuable asset: people’s trust. Years to build seconds to lose if data is ever lost
• The more users the more complicated your world becomes
  • more at stake
  • a problem that affects just 0.1% is still 25k when you have 25M
  • when the code has lots of moving parts it’s harder to add people, harder to add features. Performance optimization adds complexity to the code
• It’s not easy being lean
  • split testing & optimization is great, but you quickly run out of low hanging fruit
    • early wins with shared folder & referral flos
    • but it’s not a substitute for a great product
  • Analytics that scales with you is hard and needs dedicated engineering
  • Most needle moving factors are new features
• Build the right things and build things right
  • but if you have to choose: build the right thing
• Some design principles
  • everything should “just work”
  • don’t make users think
  • Usability, speed, reliability requires
  • don’t launch anything half-assed
    • it really going to violate the hard built trust
• It’s your job to figure out what to build, not your customers
  • Activating & retaining the users takes a lot of work
• How do we decide what to build
  • Big problems hidden in plain sight
    • For most people technolgy fails the ‘minority report’ test
      • ie: in the future Tome Cruise will not be carrying around USB drives
    • Look for unsolved problems
    • We ask ourselves how is this going to work in the future and that’s how we decide what to build
• Wrapping up
  • Building something that people love and use is rewarding. it’s worth the pain
  • A bigger audience means we can solve big problems for lots of people

3:00pm Keynote: Steve Blank
Most of the following is online @ http://www.slideshare.net/sblank/when-the-boardroom-is-bits-052111

• #1 MBA is based on 300+ years of biz
• #2 Tech Entrepreneurship + Venture capital ~ 50 years old
  • Startup are smaller versions of large companies
    • oops, not true
• #3 Large companies execute biz models, startup search for biz model
• #4 a Startup is a temporary organization designed to search for a
• #5 startups need their own tools, different from normal companies
• #6 customer developement = process to search. Tool #1
• #7 business model canvas = The scorecard – Tool #2
• #8 Agile development is how we build startups
  • startup genome projects max marmour, interview 860 startups
  • about to reveal the facts
• #9 startups that pivot 1 or 2 times
  • raise 2.5x more money
  • have 3.6x better user growth
  • 52% less likely to scale prematurely
  • than startups that pivot more than 2 times or not at all
  • startup that have
    • helpful mentors
    • listen to customers
    • learn from startup thought leaders
  • raise 7x more money and have 3.5x better user growth
• 10# we’ve cracked the code in teaching entrepreneurship
  • 54 hours later – leanstartup machine
  • startup weekend
  • y combinator
  • lean launchpad
• What we learned in the past year?
  • startup != MBA toolset
  • startup toolset is growing
  • we have real data that this works
  • a better way to teach entrepreneurship
    • Tom Heizenber at harvard
    • Nathan Furz BYU
• Hypothesis:
  • experience advise matters
  • you
    • 1 board at a time
  • VC’s
    • 6-12 boards at a time
    • x avg 4 years per board
    • x 2 VC’s per board
    • ~ 50-100x more pattern recognition
• The early stage – experience gap
• Solving the early stage – experience gap
  • venture funded companies have boards
  • angel funded companies often do not
  • experienced advice is sparse outside of tech cluster
  • people tend to leave their home/cities to go to tech cluster to get advice
• Why have a board meeting?
  • investor
    • fiduciary responsability
      • company has asymmetric info
    • maximize investment potentials
      • board has asymmetric knowledge 50-100x experience
      • good board asks hard questions
  • founder
    • board guidance
      • 50-100 more experience
      • multiple opinions across VC’s
    • board network
      • customers, partners, recruiting, financing
    • obligation
      • it came with the check
• Board meetings are heavyweight processes
  • antithesis of lean
  • board meeting haven’t changed in over 100 years
  • we act like it’s the 19th century
• why do we need to get together in one room?
• why do we need to wait a month to see progress?
• common wisdom on how to run a board meeting
  • logistics
    • send out board materials 2 days before, have board dinner
    • approve minutes, stock grants
  • business
    • summaries of key events since last meeting
  • financials
    • waterfall charts
  • management asks the board to help
    • strategy, markets, key hires, introductions
  • executive session
    • meet without management to discuss performance
  • This was the list from andrew carnegie from  1860
• What’s wrong with board meeting?
  • too far apart
  • diffuse board member attention
  • it’s often a performance rather than a snapshot
• Solution: make the boardroom bits
  • We can do this via the net
  • 24/7
  • connecting advisors to investors
  • Focus on what’s important
    • it’s about the search for the business model
      • for 20 years SB had to bring the income statement
      • first meeting brought it and it said 0
      • second said 0
      • but was formatted neatly and correctly
    • Not-income statement, cash flow
    • advice/guidance about metrics around the search
      • what are the business model hypotheses?
      • what are the most important hypotheses to test now?
      • progress of validating each hypotheses
      • what are the iterations and pivots – and why?
  • Reinvent the board meeting Continous information access
    • Founders/mgmt team invest 1 hour a week
    • structure blog
      • with unstructured data
      • to provide a narrative of your business model search
    • formal business model canvas
    • real-time advisor/board feedback
  • We made students blog their progress of customer development process with the business model canvas
  • HOW?
    • customer development
      • the process
    • Narrative
      • interviews?
        • you did one? I want to see it
      • You did a survey?
        • I want to see it
      • Videos of customer interaction?
      • prototypes
    • Business model canvas updates
      • scorekeeping
    • realtime feedback
    • physical reality checks every once in a while
      • skype
      • face-to-face
  • It changed everything
  • Competitive analysis
    • key finding of the week
    • a/b testing
    • key questions
    • strategy
  • how to keep progress we used the business model canvas every week
  • How does this change for angel-funded startups?
    • angel funded startups can now get experienced advice
      • no longer requires board commitment for involvement
      • encourages investors who need visibility
      • ends repetition for you at each investor coffee
    • turns adhoc startups into strategy-driven startups
      • invites investor participation in the process
      • professionalize the ‘throw it against the wall and see if it sticks’
    • Highly time-efficient for investor and founders
    • eliminates geography as a barrier to investment

2pm Design + Lean Startup = Lean UX

• We believe that people like <customer type> have a need for (or problems doing) <action/need/behavior>. We will know we have succeeded when <quantitative outcome>, or <qualitative outcome>, which will contribute to <KPI/success for our company>
• todo
  • go to a lean UX meetup
  • recruit UX and cut them some slack
  • learn to do it yourself
  • train together and then collaborate
  • go on a site visit with your designer
  • luxr.posterous.com
• the ladders
  • techniques
    • style guides
      • get the mundane out of the way
    • asset library
    • Collaborative problem solving / design studios
      • Outcome, not output
      • create just enough design to get started
    • Pairing designers with developers
      • at the end of the session you have an experience, not a wireframe or documentation
    • learned : human to human > paper to human
    • big learning 2: personality in designers is key to success
      • stay open to new ways of working together
      • transparency builds trust
• Designers
  • Designers do customer development. They call it research.
  • Designers want to understand how users think about a problem.
  • Design research reveals what users value
  • Designers need organizational support
  • Designers who know how to collaborate are the ones who win
  • Designers work at every layer of the stack, from strategy to visual.
• Zach Larson – sidereel.com
  • Information radiator
    • information up on the walls
    • public visible
    • in a highly trafficked area
    • =push instead of pull
  • Theme backlog
  • ux prototypes
  • What’s a successful information radiator?
    • information light
    • cheap to build and maintain
    • informs discussion so that conversations are richer
  • the big rewrite
    • exposed problems
      • home base didn’t know what the product team was up to
      • developers didn’t always know the most recent insights
      • It was hard to define UX
    • Radiators solved the problems
    • Sketched ideas/user flows
    • for big picture stuff too
      • themes(with just enough detail) ex:
        • Theme title: Social profile pages
          • goals
            • user registration, community
          • Public version of profile pages to collect user actions and maybe share tracked shows
          • see friends with connected accounts
          • personalized urls
    • answer basic question, make the most important info easy to access
    • When to make an IR
      • when folks keep asking for the same information
      • when there is a core set of info to share
      • when the people who see it will share a context
      • only use it when people find it useful

11:50am IGN entertainment case study “BigCo lessons learned”

• 5 whys
  • make sure that you do something about it.
  • Takes practice to do them well
  • Somebody needs to be in charge/responsible of the process.

11:30am food on the table one year later

• Concierge MVP
  • Literally have a meeting with your first customers and hold their hand through the process.
  • Go through the whole process & validate the process / product
  • If the CEO is there and with the reality distortion field. If she can’t get excited/convinced about the product, it’s not going to happened on a cold website.
• Key
  • keep in mind of the grandiose idea but go back and distill it into the basic idea.
• Scaling down
  • Stores
    • Big plan
      • top chains in 50 states
      • every store per chain
      • all sales items per store
        • => 35,000 stores over 1.5M sales items /week
    • MVP
      • only care about 1 store
      • madue sure that our first 10 users shoppoed at that store
      • protein drives dinner decision
      • 1 store => 5 sales items
    • Today
      • 13,000+ stores across 50 states
  • Recipes
    • Big plans
      • all main ingredients/cooking methods/etc
      • => 100,000 recipes
    • MVP
      • the right recipes
      • 5 recipes per ingredient
      • Progressively add recipes for new sale items
      • => 20 recipes
    • today 30,000+ recipes based on actual needs
• Usability
  • Big plans
    • Many ux analysis/flows/IA
  • MVP
    • 5 paper prototypes pages
    • worked on paper for a while until we worked out the kinks before making the expensive investment to develop
    • We used google docs spreadsheets and copied and pasted the information manually for each customers.
• What we learned
  • break down learning into small steps
  • once you start anticipating user feedback, you are ready for the next step
  • learn first, code last
• Pay attention to your customers
  • I want to do my meal plan in the parking lot of the grocery store
  • Active users on mobile
    • within few weeks of mobile app launch, half of the users are mobile.

11:00am Keynote: A conversation with intuit CEO Brad Smith

• 6 core values out of which
  • Hire & test for innovate & improve
    • innovate & improve anything you touch
  • by observing the founder, who still works at intuit, they took his instinctual actions, what he does naturally and tried to make it a process
• Methodology that we extracted from the founder
  • team that are no bigger that 2 pizza can feed
    • more then 6 people, somebody needs to get out of the room
  • idea to concept in 6 weeks or less
    • The rougher the prototype the better you are
  • The corporate gov cannot impose more then 3 policies/rules/restrictions on the team
    • corporate is there to remove barriers not to help
• 10,000 hours of follow me home
  • just observe the users as they use the product
  • almost everybody in the company does this getting out of the building and customer observation including the CEO.
• asking the right questions at the right time
• environment for experimentation
  • expose the product to the customers as soon as you can.
  • allow the entrepreneurs in the company to be creative
• Hire entrepreneurial engineers
  • somebody that wants to put a dent in the universe
• We don’t fail if we didn’t invent it
  • pick a fight with the new guy
    • every quarter:
      • tell me 3 companies that I haven’t heard and what can we learn and adopt from them
    • If we can’t beat them let’s see if we can absorb them
• genius with a thousands hats doesn’t scale

• asking the right questions at the right time (in the product’s life) is key to the success of the project.
  • conversely the wrong question at the wrong time can destroy the product/project

• • The question that we ask
    • in early startup we ask the love matrix
      • what’s the 1 reason the customer buys the product
      • have you nailed the customer’s benefit so well that he’s going to tell friends & family that they got to use the product
      • have they started actively using it in their daily life
    • We don’t ask
      • revenue,
      • are we going to make money
      • are they ever going to pay us
    • We’ll figure out monetization later
• Horizon planning
  • Horizon 1
    • existing businesses
      • growing category
      • increasing word of mouth
      • increasing market share
  • Horizon 2
    • adolescent businesses
      • keep going for top line growth
      • introduce financial metrics
  • Horizon 3
    • the 2 people team
      • lovematrix
• Truth & worth – criteria to distribute bonuses
  • are you bringing the talent & the engagement scores that we’re looking for
  • are you growing customer base
  • increasing market share
  • portfolio at age 3
    • # of customer that are using the product at age 3 that didn’t exists 3 years ago
    • % of revenue that you’re producing that didn’t exists 3 years ago.
• How do you invest/allocate money in all the parts of your company?
  • They take an horizon planning approach
  • Intuit is $3.5 billion company $2.1 Billion in operating expenses (or investment)
  • 60% Horizon 1
  • 30% Horizon 2 – scaling adolescent businesses
  • 10% carved out whether it’s time/people/$
• They need to open up their technology
  • because while they are 8000 passionate people working at intuit
  • they have 50 million passionate users with also great ideas and insight
  • 3rd party developers
  • ie:
    • live community
      • live chat inside of turbotax
      • 40% of all customer questions got answered by community members with higher accuracy that intuit’s own customer service could answer. Including question that for legal reasons couldn’t be answered. (sounds familiar to Hedgeye situation)
    • applied the same concept to quickbook
      • people are contacting each others for counseling, help, suggestions.
• brad_smith@intuit.com

 

10:40am wealthfront case study

• Continuous Deployment
  • reducing the lines of code/ code inventory
  • traditionally
    • product goes in development
    • product manager cuts a release
    • then it goes in staging
    • qa
    • fix bugs
    • integrate patches
    • release
  • google maps releases every 2 weeks
  • revisited
    • cutting a release is not an event since it’s always in working order
    • no staging
    • testing the whole systems in < 3 minutes
    • p1 bugs & patches are non existent because the code is either in good or bad state
    • no releases since the code is in deployed continuosly
    • ++ experiments
      • Way to show/hide features or release in stealth mode
    • ++ monitoring
      • we need to be informed in realtime
    • The goal is to get the deployment under 5-10 minutes
• deployment manager
  • latest commits
    • are they good or not
  • statuses of the build
  • statuses of production
  • statuses of current deployment
  • recently completed releases
• 5 essentials
  • culture
    • empowering the engineer to define functionality and own quality
    • implement the vision – ambiguous – on automatable
    • make sure it doesn’t regress – automatable
  • test driven development
    • to make sure regression are caught
  • continuous integration
  • immune system
    • production pager/monitoring
    • warns an engineer if something deviates
  • continuous deployment
• Pascal-Lous Perez
• http://eng.wealthfront.com/2010/05/deployment-infrastructure-for.html
• http://eng.wealthfront.com/2010/02/culture-is-key.html
• https://github.com/wealthfront

9:50am keynote a Conversation w/Mitch Kapor 

• investing not just in companies that have profit value but also in companies that provide social value.

Watch live video from Startup Lessons Learned on Justin.tv

10:20am votizen case study

• Why lean?
  • mitigate your own reality distortion
  • helps avoid expensive false positives
  • converts market risk to technical risk
    • market risk: is somebody going to buy this
    • Technical Risk: how to create the thing
• On Pivots
  • not synonymous of change
  • restating your business model
  • pivots are vision driven, not testing driven
  • Pivots are a consequence of learning about your vision
• Summary
  • Set Model so you kow what you’re measuring and when you’re done
  • Use Metrics to find local maxima, not to find a business.
  • Pivots are vision-driven.

Watch live video from Startup Lessons Learned on Justin.tv