The Ning Infrastructure Roadmap

The Ning Infrastructure Roadmap
5 (100%) 1 vote

The Ning infrastructure runs all Ning Networks. It consists of thousands of machines with software that scales to millions of users. We have consistently delivered high uptime with our platform. As the usage characteristics change, the behavior of our infrastructure changes as well. We keep working on a number of projects that are designed to evolve our infrastructure and make the platform even more stable. As I promised last week, I wanted to provide details of the infrastructure projects we are currently working on. They’re not visible to Network Creators and are all low-level “plumbing” work, but they will enable higher uptime for all Ning Networks.

The first major project is Project Bacon (note: I am a vegetarian). Project Bacon will be the next generation “Content Store” and will give us much better fault isolation and stability for all the content on Ning Networks like comments, forum posts, blog posts, etc. Let me go over some details so you have context. The Content Store is the most heavily taxed component in our platform. It holds photos, blog posts, comments, and other information for your social network. We currently have two different back-ends for the content store, called Rocky and Coco. Rocky is optimized for our largest networks and Coco for our smaller networks. While running both has been good in terms of optimizing the solutions for different problems, it has not been so good in that it means double the work for adding features, and a much larger surface area for bugs to creep into. Taking what we have learned about building these (and they are the third and fourth generations of the Content Store), we are re-unifying the back-end with Project Bacon. While most of Bacon is an evolutionary step in terms or organizing and accessing data, a key component is much better fail-over and fault isolation in cases where we do see issues in its constituent components.

A second big project we have underway is revisiting how we store user-to-user messages and invitations. For efficiency reasons we have kept all of these in one big storage pool, but the fragility of this approach has started to outweigh its efficiency benefits. If you think of the single large pool as a window with a single pane of glass, a rock hitting the pane can shatter the whole thing. On the other hand if you have a window with many small panes of glass, a rock hitting one is not going to hurt the rest of the window.

Another major undertaking is supporting much better automation in our system, from development through to production. We’ve just started this but expect to see incremental fruit from it very quickly. Jim Gray discovered 25 years ago that the largest causes of failure in a system (about 42%) are operator error, misconfiguration, and system maintenance. Automating away the human-intensive (and therefore error prone) steps here have outsized returns, so we are revisiting our environment build-out, deployment, and configuration systems to remove room for errors.

Finally, we are finishing a long process of transitioning our caching infrastructure away from a more sophisticated and cluster-aware solution onto memcached. While memcached offers fewer features, it is also inherently more stable and predictable. This has been a long process to see all the way through as it meant reworking some components which relied on features not available in memcached. On the other hand, memcached’s architecture leads to better predictability and performance at large scale.

Overall, in these projects and others, our goals are:

• Simpler internal systems
• Removing components which add more problems than value
• Isolating failures to as few networks or users as possible
• Limiting failure duration to as short as we can

No related posts.

15 thoughts on “The Ning Infrastructure Roadmap

  1. Ken Carroll

    Hi, I’ve only just discovered this post. It provides a useful insight into how much work is going on at NING ‘behind the scenes’. Thanks for sharing this, would be interested in hearing how its progressing since your post is dated May 2010.

  2. David Benson

    Hello Staff,

    I am frustrated that I can not upload a music or audio file within a blog post or forum post on my ning plus site. You give to me a Muzicc App, but that is not what I desire, and it stops playing when I change pages on the site. That Muzicc App would be a great background filler for anyone desiring to listen to music of their choice while looking at photo albums to going to a blog or forum post. If perchance an audio file was initiated, then by all means cut off the muzicc box. How can I get to post an audio file to my site, and yes on the old ning site Iwas able to post my own video productions, now it seems that if it is not YouTube.com then it does not get posted,.

  3. Marc Emerson

    Hi Sridatta
    looks like an sanyasin name direct from POona (OSHO)
    Happy to read about the work you are doing in linking rocky and coco ,a merge to help all creators to enjoy our networks regardless if big or small
    Please tell me where I can find info as to set up a ning network in french and one in english ,different but linked to same designers ,can I merge one name and ahve to different networks or do I have to register two different names
    Thank you
    Marc emerson

    1. Eric Suesz

      Hello, Marc! We have a site where folks can ask questions of a community of people called the Creators Ning Network. We also have a Help Center that has a lot of informative articles about this. You can also submit a ticket in the Help Center if you are having specific problems and we’ll try to get you some one-on-one help. Hope this helps!

  4. JamesR404

    Wow! What a bunch of offtopic reactions, guess it shows how diverse your readership is 😉

    I for one am happy to get an inside peek in how Ning’s systems are maintained and evolved =]

  5. Chad

    Just wondering when you guys plan to launch the paid structure. Wanting to get some premium services but it looks like they will be in the monthly sub package I plan on getting.

    Thanks, looking forward to the new launch because the free sites are great even, been able to do a lot of practicing while waiting. Great job!!!

  6. Kirlisa Broxton

    Hey guys,

    I’m a new member and I wanted to see if you know how to set up your profile

    1. Eric Suesz

      Hi, Kirlisa. We have a section for Network Creators called Creators where you can ask questions if you need help. You can also submit a Help Center ticket if you want some one-on-one help.

  7. Luis Gubler

    Thanks for the info, but I believe a “product roadmap” would me more useful,

  8. Jeff

    The ning network service is fantastic! I use it to help musicians market themselves and connect with other musicians. I do have one request… Please, change the event section where you can add multiple events. I work with clubs and musicians and it is a huge pain to add events one at a time. There is a bunch of people on the developer forum who concur.

    Other than that, Ning is a huge timesaver and help! Go Ning!

  9. Lesley Dewar

    While this is going on, I am experiencing recurring failures while trying to embed pictures in my blog. Being “welcomed back” to your damn homepage in the middle of a blog update on my network is NOT a welcome interruption of my work.

    1. Eric Suesz

      Hi, Lesley! This is the blog, so I can’t really help you here in the comments section of the Ning Blog, but if you submit a Help Center ticket, we’ll get someone to look into your particular issue. Thanks!

Comments are closed.