A historic moment in history has occurred with the election of Barack Obama as the next President of the United States. For many, this moment was shared with the digg community that included a furious level of digging and commenting. In fact, election night generated the most traffic and activity in the history of Digg.

At about 8pm PST, most TV networks called the election with Obama as winner. Some interesting stats for the 8pm hour (PST):
Submitting was 108% of normal.
Digging was 202% of normal.
Burying was 137% of normal.
Commenting was 278% of normal.
Comment Digging was 619% of normal.
Comment Burying was 689% of normal.

Note the resulting traffic on one of our DB chains:


Wow. Talk about making the DBAs sweat. If you were on digg at this time you might have noticed a couple of annoying issues. Logging in wasn’t pleasant. Page load times were longer than usual. Digging was… a bit… unresponsive, submitting a new story may have been prone to failure.

If you recall the previous blog from our lead DBA timeless, you’ll remember we have a concept of ’selectors’ where developers will select a pool and purpose for their db related code. An example might be “main_write” as a selector. This would give a handle to the main database chain for the purpose of updating or inserting a row. A handle like “main_read” would give a handle to one of the many slaves of our main master db. Transactions going to “*_write” are supposed to be very quick and cheap. Our write masters do not run the dbmon.pl software to kill off connections – that is reserved for the massive pool of read slaves.

Now, here’s the bug. A small number of our queries on “*_write” are not writes. They are reads for a very, very small subset of queries that need the absolute latest information. Any slave lag might cause weird problems with how we keep certain elements (like new user handles) unique. Unfortunately, a bug in our code that does this ‘quick check’ was generating 6 or 7 thousand quick checks… Multiply this by the huge amount of traffic Obama generated and our ‘main_write’ selector ran out of connections:

While the graphs show problems - they are only clues an investigation needs to start. For the above, I parsed all the queries from the hour before as a baseline measurement of ‘normal traffic’, and then took a look at the 8pm hour. Most of the work is in figuring out ways to classify and group similar queries together to pinpoint anomalies. Often after grouping everything together, it is quite obvious that a certain class of queries is causing problems. For example, we were seeing a very disproportionately high number of queries of one class in relation to other classes of queries. Once the issue had been isolated, I sent the results via email to one of our software engineers. In this case, Kurt, looked at the anomaly and tracked down this bug introduced some time ago.

After pushing the code we have seen much more consistent normal traffic:


Thanks to Obama, the many people who took interest in the elections, and our digg community, we have been able to fix the bug. We’re always grateful for your participation and feedback on Digg. It’s incredibly helpful. So, keep it coming and stay tuned as more improvements are on the way.

Digg on.

Driver

[Digg Technology ]
View original post|Add to del.icio.us | Share

Hey guys -

The Digg team is heading back to Mighty in San Francisco for our last Digg Meetup of the year. Come raise a pint, take a picture or two in the Photoboof, and hear some cool announcements - the perfect way to spend your Wednesday night. In case you haven’t had a chance to attend one of these events, check out some of the footage from our Meetup a few weeks back in London.

<object width="400" height="300"><param name="allowfullscreen" value="true"/><param name="allowscriptaccess" value="always"/><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=2246107&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/><embed src="http://vimeo.com/moogaloop.swf?clip_id=2246107&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>

We’re also excited to announce some more dates and cities for future Meetups. In the first few months of 2009, we’re heading to LA, Austin, and Seattle - details on the venues and other 2009 events coming soon. As always, for all the latest up-to-date information, make sure to keep an eye on our Meetups page.

Looking forward to seeing you all soon,

Kevin

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey Everyone,

Our next Townhall meeting is Tuesday, November 18th at 8pm ET/5pm PT and we’d like to hear from you. Similar to past Townhalls, you submit and vote up the questions and topics you would like to see answered. Jay and I will then address the most popular questions from the community live via webcast here on the Townhall page. It will also be available to view anytime afterwards.

It’s been a crazy few months here at Digg and I’m sure we’ll have lots to talk about. So, please submit and digg up your favorite questions in this thread.

Looking forward to a great discussion,

Kevin

[Digg Company ]
View original post|Add to del.icio.us | Share

The newly remodeled Digg Shop is now open for business! We’ve updated our most popular hoodies and tees, and added new styles, colors and other cool stuff. Also added are laptop messenger bags, 1337 Digg t-shirts and boxer shorts.

If you have more good ideas for new t-shirt designs, we want to hear about them. So, post your designs and ideas in the comments.

Thanks,
Matt

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey All,

Last week, former Vice President and Nobel Prize winner Al Gore was our second Digg Dialogg guest. For those of you who didn’t catch the program on Current TV this weekend, you can watch it online here and at Current. The Digg community submitted over 1,000 questions, ranging topics from the election to Al Gore’s Twitter account. We’ve included a quick highlight below (the full version is available at the links above).

<object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" width="400" height="400"><param name="movie" value="http://current.com/e/89504392/en_US"></param><param name="wmode" value="transparent"></param><param name="allowfullscreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed type="application/x-shockwave-flash" src="http://current.com/e/89504392/en_US" width="400" height="400" wmode="transparent" allowfullscreen="true" allowscriptaccess="always"></embed></object>
Last week’s election was definitely one of historic proportions and many folks joined us for the Election Night party, hosted by Digg and Current. It was Digg’s highest traffic day in history and a historic day for democratized media. I want to personally thank all of you for helping Digg and the community be part of such a memorable presidential election.

Thanks,

-Jay

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey Everyone,

A few months ago we successfully kicked off Digg Dialogg with Nancy Pelosi at the Democratic National Convention in Denver. For those of you who missed it, Digg Dialogg gives the community an opportunity to submit and vote up questions to be posed to influential individuals and leaders of the moment. These questions are unfiltered by editors or journalists, and represent the most popular questions as voted on by the community.

I’m excited and honored to announce our next Digg Dialogg guest, former Vice President, Nobel Peace Prize winner and Chairman of Current Media, Al Gore. I can’t think of a better time to get the perspective of one of the most influential leaders of the decade. Topics can range from politics to the environment to the economy - it’s really up to you.

On Wednesday, November 5th, on the heels of the Current TV and Digg election party and the results of the 2008 presidential race, we will open up Digg Dialogg for questions. Stay tuned for the homepage link when the questions go live at midnight PST. The interview will premiere on Current TV at 10:00pm EST/PST this Friday night, and will be available online by 11:00PST. We will also post regular updates to the Digg Twitter account.

We’ll be announcing more guests soon. As always, let us know who you’d like to talk to by adding suggestions in the comments.

Digg on,
Kevin

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey everyone,

Many of you have been following election news on Digg, including our visit to the conventions last month, Digg Dialogg with Nancy Pelosi, and regular updates to Digg the Candidates. The upcoming US Presidential Election this Tuesday, November 4th will be one of the most exciting and closely watched in recent history. We want to give you the opportunity to share your perspective on the election in a creative way, so we’re joining forces with Current TV and Twitter on election night to bring you real-time results and Digg headlines, aired live on Current. Basically, it’s an election viewing party on TV, with the sounds of DJ Diplo as a soundtrack.

<object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" width="400" height="400"><param name="movie" value="http://current.com/e/89467344/en_US"></param><paramname="wmode" value="transparent"></param><param name="allowfullscreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://current.com/e/89467344/en_US" type="application/x-shockwave-flash" width="400" height="400" wmode="transparent" allowfullscreen="true" allowscriptaccess="always"></embed>
Check out Current on election night. If you are in San Francisco, come watch the results and raise a pint with the Digg crew at 330 Ritch, between 5:00pm and 10:00pm. I will also be providing a few live updates from the party, along with Martin Sargent.

Digg on,
Kevin

PS: You can also check out a rough video of some of the Digg crew’s convention activities here.

[Digg Company ]
View original post|Add to del.icio.us | Share

Allow me to introduce myself. I’m Paul, Sr. Systems Engineer and resident Puppetmaster for Digg. “Puppetmaster?” I hear you say, “What do puppets have to do with Digg?” Well, that’s what I thought I’d write about today.

To answer the question, it isn’t “puppets”, it’s Puppet, the open-source configuration management tool, and a relatively new addition to the Operations Secret Sauce here at Digg. Dr. Timeless and Joe have already given you a good overview of the complex architecture that makes Digg possible; in my posts I’m going to go into detail on how we manage some of the components of that architecture using Puppet (and touching on other tools along the way).

First off, let me answer a couple of questions that I’ve had to answer in the past. Why configuration management? Why Puppet, specifically? Configuration management buys you a number of benefits, and other people have written about those benefits more extensively, and more eloquently, than I can. Suffice it to say the Ops team at Digg believes configuration management is an incredibly powerful tool for managing complexity in a large-scale architecture (which, by complete coincidence, is exactly what we have!)

We chose Puppet after evaluating a number of other options (cfengine, and bcfg2 among others). Like many people, we thought that Puppet, cfengine, and bcfg2 were the top contenders in this space. Cfengine is, in a way, the grand-daddy of open-source configuration management tools. Unfortunately, certain design decisions and philosophical stances it assumes leave it somewhat behind the curve. Bcfg2 showed a lot of promise, but in the end it lost out due to a few key factors. First, there was a lack of “Bundles” supporting the things we needed to manage; Puppet had “types” which more closely matched our needs. Second, the philosophy of Bcfg2 is a “total management” philosophy; you can’t deploy a Bcfg2 configuration that only manages a small portion of a machine’s configuration. For our needs, the ability to implement configuration management incrementally was very important, and Puppet gave us that flexibility. Third, the overall design philosophy of Puppet makes a lot of sense to us. Last, and least importantly, I had more experience with Puppet that I could draw on as we rolled it out.

For those of you that are unfamiliar, Puppet is a multi-tentacled beast project with several components: a declarative language for describing system configuration, a standalone parser for that language, a resource abstraction layer for providing platform-agnostic manipulation of the resources described by the language, and a client-server daemon for distributing and applying configurations described by the language. (Phew, what a mouthful!) The foundation of Puppet’s power is the resource abstraction layer, which I will talk about below. The rest of the Puppet components (and how we use them) will have to wait for another post.

Puppet’s resource abstraction layer (the RAL, to save me some typing) allows us to think of various aspects of system configuration as “resources”. Puppet comes with a tool called ralsh which allows you to interact directly with the underlying RAL; you can use it to list resources or manipulate them in the same way Puppet does. “But what exactly is a resource!?” I hear you cry. A resource is a discrete component of system configuration. Some examples are: cron jobs, users, host file entries, mount points… Puppet’s RAL knows how to manage a wide variety of resources natively; it is also fairly easy to extend it to manage resources it doesn’t already understand.

ralsh is an incredibly powerful tool, and decidedly underused in the Puppet community. Let’s pretend you’ve inherited a decidedly non-homogeneous architecture, filled with various flavors of Linux and *BSD. Want to add a user? Just remember that on linux you can use adduser, or useradd, but on FreeBSD you probably want pw, and who knows how many different command-line switches you need to remember, or maybe just consult the man pages every time… Or, use ralsh. It provides an interface that works on every platform Puppet works on. The interface is the same on every platform. Want to know what users are defined? ralsh user lists them. Just want to know about this guy named baduser? ralsh user baduser will give you the dirt. Want to add a user? Watch this:

~$ ralsh user newuser uid=9999 gid=9999 home=/home/newuser shell=/bin/bash ensure=present
notice: /User[newuser]/ensure: created
user { ‘newuser’:
uid => ‘9999′,
gid => ‘9999′,
home => ‘/home/newuser’,
shell => ‘/bin/bash’,
password => ‘!’,
ensure => ‘present’
}

This doesn’t get really cool until you realize you can use the exact same tool and syntax to manage any type of resource that Puppet understands. Check it out:

~$ ralsh package xmlstarlet ensure=present
package { 'xmlstarlet':
ensure => '1.0.1-2'
}

~$ ralsh cron check_my_mail command="/usr/bin/fetchmail mail.my.mail.server" user=plathrop hour='*' minute='*/30' ensure=present
notice: /Cron[check_my_mail]/ensure: created
cron { ‘check_my_mail’:
command => ‘/usr/bin/fetchmail mail.my.mail.server’,
monthday => ‘absent’,
hour => ['*'],
environment => ”,
target => ‘plathrop’,
special => ”,
minute => ['*/30'],
user => ‘plathrop’,
ensure => ‘present’,
weekday => ‘absent’,
month => ‘absent’
}

This is totally cool! Suddenly I don’t have to care what platform I’m on, and I can think of these things in an abstract, encapsulated manner. I can choose platforms based off of what they are good at instead of requiring homogeneity in order to minimize the costs of management. Like OpenBSD’s firewall, but you have a Debian network? By all means, throw an OpenBSD box in there; you can use the same commands to manage both. Want the performance of a FreeBSD network stack for a certain application? Go for it, you already have the tools you need to administer it.

Not only that, but if you are building a Puppet infrastructure, you can use ralsh to explore your existing configuration; the output you see above is valid Puppet code, and can be used in a Puppet configuration as-is!

Next time I’ll talk more about the higher-level components of Puppet; the RAL, though awesome, is just the foundation. The Puppet language allows us to treat configurations as code; we can use the same techniques software engineers use to manage their codebases to manage our systems configurations. The Puppet client/server allows us to apply consistent configurations across a cluster of machines, and manage the entire life-cycle of each system, from deployment to retirement. Combined with a flexible and robust automated deployment system like Debian FAI, Puppet can help you drastically reduce the intervention required to bring a machine up from bare-metal to production; saving both time and money as well as giving you a chance to focus on more important issues.

See you next time.

–Paul

[Digg Technology puppet ralsh ]
View original post|Add to del.icio.us | Share

Hi all -

Later today Digg will be undergoing some significant updates, but upon first glance, it may seem that nothing has changed. That’s because the changes included in the coming release are 100% under the hood. For the last few months, Digg’s developers have been heads down scrubbing, cleaning, porting and improving the core code of Digg. Before I get into the details, I want to give a big thanks to everyone on Digg’s development team. They’ve done a great job with what, technically, is one of the largest pushes Digg’s released.

Digg’s development team has grown tremendously over the last few years to over twenty developers (we’re hiring!). At the same time, Digg’s code has been maturing from a monolithic ad-hoc approach to a set of consistent frameworks. Our goal was to standardize the way applications at Digg are written, managed, and deployed. Sounds fun, but how did we do it?

- During the profile redesign, we also rewrote our login and registration system using an internal event driven framework we call App. It’s a small, simple framework for quickly creating applications. App implements the ‘V’ and ‘C’ in ‘MVC’ along with input sanitization and authentication. Basically, it allows Digg developers to rapidly develop applications without worrying about the basics. As of this latest push we’ve ported the majority of Digg’s applications into App. Every developer at Digg has had a hand in porting thousands and thousands of lines of code to App.

- We also have another framework, called AJAX (I’m not very creative at naming my frameworks), that manages all of our AJAX endpoints (the PHP code that processes and responds to AJAX calls from jQuery). AJAX allows our developers to create a simple PHP class to process requests while the AJAX framework handles JSON and AHAH encoding, token checking, authentication, XSS/CSRF checks, input sanitization, HTTP error codes, exceptions, etc. Before this push about 70% of our endpoints were in the AJAX framework. As of this forthcoming push we’ve managed to port 100% of our endpoints to AJAX.

- One of the major problems at Digg, with regards to development processes, was that our code was monolithic. With three developers this isn’t a problem, but with 20+ it can get hairy. It can make merging difficult, interdependencies impossible, and make it impossible to promote/enforce ownership within SVN. In response to this, all of our core frameworks, applications, etc. are now PEAR packages. For those who don’t know, PEAR is a package management system for PHP, along with being a world class repository of PHP libraries. This allows us to break our code up into dozens of smaller projects which are owned by individuals and teams. PEAR also allows teams to define specific PEAR, PHP and PECL code that it depends on. What this means is that App_Login can enforce dependencies on App, Message, Mail, etc. and deployed separately from App_PermaLink or App_StoryList.

- Since our VP of Engineering John Quinn has joined the team, he’s been spoon feeding us the Agile pixie dust. Part of the Agile philosophy is unit testing, and the result is that all code at Digg must now have unit tests which are fed into our continuous integration environment. Our CI environment automagically checks out code every ten minutes, runs unit tests and verifies the code conforms to our coding standards, and then emails developers of any problems.

So, despite the appearance of nothing changing, big changes have been afoot in Digg’s development department. Tens of thousands of lines of code have been rewritten, massaged, ported, unit tested and modularized. But, why? So we can continue to develop quality new features at a quick pace even as we continue to grow our development team.

Thanks!

- Joe

[Digg Technology ]
View original post|Add to del.icio.us | Share

In the next week or so, we’ll be closing down the podcasts section and folding it into the video section of Digg. We’ll also be retiring the old Digg Spy. Both of these features have become outmoded as Digg has grown and as a result they have a very small number of users (under 1,000) each. Podcasts will be rolled into the videos section – a better home for longer form videos. Digg Spy will be retired as we look to implement more exciting new Digg Labs projects in the future.

Recently I’ve been speaking at several design conferences about the necessity for iterative development. One of the key points I’ve been making is that subtraction is an oft-overlooked type of improvement. The retirement of these two features on the Digg site is an example of this principle in practice.

DIGG PODCASTS

The podcasts section experiment started almost two years ago. At the time, podcasts were relatively new and we saw them as a unique medium – different enough from audio or video to warrant a separate and custom-designed section of the site. That section was developed to differentiate between each ‘episode’ and the parent ‘show’, which could be ranked over a long period of time. Unfortunately, as we all learned, the podcasts section stagnated because the top shows dominated and there was little activity. This shortcoming is one reason that the podcasts section is used by less than a thousand people on a regular basis.

Over the past couple of years, we also saw a rise in the submission of episodic videos in the main sections of Digg under the ‘video’ media type. Television shows, individual podcast episodes, and clips from shows were frequently intermixed in the main Digg river. This type of natural activity, which can compared to the idea of desire paths, is a pretty strong indication that ‘videos’ is a better home for podcast-type material. By eliminating the awkward differentiation of podcasts, we’ll be greatly simplifying Digg from a user standpoint.

DIGG SPY

The original Digg Spy is being retired for different reasons. We built Digg Spy in the very early days of Digg, when the activity level was less frenetic than it is today and we could show a lot of the action through a live activity stream. It was great. You could discover new content and new people in a visual way that AJAX was just then letting us do.

As Digg grew, Digg Spy became less and less representative of the breadth of activity on the site. We began showing ever-smaller percentages of the activity on the site in order to keep the stream from becoming a blur. Thus, Digg Spy became less informative. When we added the Big Spy feature to the Digg Labs, a better and more entertaining version, the value and distinction of the original Spy became even less clear. We therefore elected to remove it as a feature that’s outlived its purpose. Digg Spy is dead, long live Big Spy.

One issue I suspect may be brought up is that Digg Spy is one place on the site that surfaces some burying activity. People have tried using Digg Spy to track burying activity and I won’t be surprised if conspiracy theorists accuse us of burying (pun intended) the feature to hide this. In fact, only a very small subset of buries on the site actually appeared on Digg Spy due to the small window of activity that was actually visible through the feature and any ‘patterns’ that people perceived by watching the buries have always been grossly inaccurate.

SUMMING UP

Occasionally pruning is the prudent thing to do – in these two cases, I’m confident that it’s the right course of action for the longer term vision of Digg. Thanks so much if you’re one of the people who regularly visited the podcasts section or if you enjoyed watching the Digg Spy stream past – we really appreciate your participation.

Cheers, Daniel

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey everyone,

The Digg crew is coming to London this very Friday, as part of the closing party for the Future of Web Apps conference. We’ll be at the Fox from 8:30pm ’til midnight, so come out and say hi if you’re around.

We’re partnering with FOWA London and Revision3 for a live Diggnation directly followed by a Digg Meetup, sponsored by Digg and Facebook. It’s free to attend and open to everyone (not just FOWA attendees) so come raise a pint with fellow Diggers for a great evening. Stay tuned for Digg updates, and DJ Soul of Man from Finger Lickin’ Records will be spinning records too.

It was great to meet some of you at the last Chicago Meetup. We put together a quick video of some highlights…

<object width="400" height="300"><param name="allowfullscreen" value="true"/><param name="allowscriptaccess" value="always"/><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=1871002&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/><embed src="http://vimeo.com/moogaloop.swf?clip_id=1871002&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>

We are also planning upcoming Meetups in San Francisco, LA and Austin. Check out our Meetup page for the latest details.

We hope to see some of you out there!
Kevin

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey all –

Digg enforces its Terms of Use so that Digg remains a vibrant community of people committed to sharing and discovering great content. Everyone who uses Digg agrees to abide by the TOU, which maintains a positive experience on Digg for all community members by prohibiting spam, porn, gaming, hate speech etc.

In many cases, Digg gives users who violate the TOU a second and sometimes even a third chance. When people continue to violate the TOU, or where a first-time violation is egregious, Digg is reluctantly left with no option but to ban the user.

A couple weeks ago we posted a blog entry regarding script usage, reminding members of the community that it violates the Digg TOU. We also rolled out some changes that warned users when unnatural Digging activity was detected. Since this post, we analyzed our logs on a regular basis to clearly identify script use over an extended period of time.

While we never speak to specific instances of user bans to protect the privacy of individual users, we have banned a small number of users for script use over the past several weeks. Some of them are active users that are well known within the Digg Community. While we’ll sincerely miss the contributions of these individuals and are never happy about playing policeman, we believe that the larger Digg community is adversely impacted by people who choose to violate the TOU.

Please don’t hesitate to contact us at support@digg.com with questions or feedback. We’re continuously researching and investigating this process, so don’t be shy and let us know what you think.

Have a good one,
Jen

[Digg Community ]
View original post|Add to del.icio.us | Share

Calling all Diggers,

If you’re keen to work at Digg, here are 8 solid reasons to send us your resume right away:

8. You’ll Digg the DigDug
When Kurt isn’t solving complex infrastructure problems, he’s working on his DigDug high score. Think you can beat it? Think again. And while you’re thinking, how about sending in that resume? 7. Reduce your CO2 footprint
Digg is bicycle friendly. So why not help save the planet while getting a workout? Digg is easily accessible from most of San Francisco or from the peninsula via Caltrain. What’s more, you can park your bike by your desk. So hop on your fixie, cruiser, or pink folding commuter with a basket and handlebar tassles and pedal on over to Digg. 6. You can snuggle up to the Digg pets
The Digg office has dogs aplenty and the occasional visit by “Pistol”, Digg’s very own guinea pig. Mmmmm, yummy little guinea pig. (Please don’t let on to Jim that we know Pistol is really a ferret.) 5. We’re located in San Francisco’s best neighborhood
Digg is located in San Francisco’s Potrero Hill neighborhood, boasting one of the best micro-climates in the city and lots of great places to eat and drink. Our local coffee shop, Farley’s, claims that it’s San Francisco’s best independent coffee house. Perhaps, but just don’t ask for a low fat latte - you won’t be welcomed back. :) 4. The fridge is always well-stocked
The Digg fridge is kept well-stocked at all times. And don’t forget Digg’s weekly “Wine Wednesdays.” 3. You’ll grow to love your MacBook Pro
Ah, the simple pleasure of an MacBook Pro and a 30″ monitor. Now if I could only figure out where my right mouse button has gone… 2. Who can’t use 20 PTO days?
At Digg you’ll enjoy 20 PTO days, 11 paid company holidays, a great health plan and a wellness (gym) reimbursement. 1. We Rock
The best reason to come and work at Digg: we rock. Or we try to. Don’t take my word for it - watch the video.

Digg is hiring in Development, R&D, QA, Product, and other areas. Check out the jobs page for details.

Thanks,
John

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey All —

Today is a big day for Digg. We’re announcing a major expansion effort – the largest we’ve undergone in our history. With a new round of funding, we’re accelerating many of the programs that we’ve been working on over the past several months, including investments in infrastructure, new feature development, international expansion and hiring all the people we need to get there.

You’ve given us some great feedback on how we can make Digg better. As you’ve heard us say many times before, we feel that we’ve only implemented a fraction of the vision for what we believe Digg can achieve. The new features that will be a part of this program will incorporate much of your feedback, including personalizing the Digg experience, enhancing the recommendation system across other areas of the site, creating deeper category and topic content views and more ways to discover and organize content.

Of Digg’s over 30 million unique monthly users, almost half are from outside of the U.S., so this expansion will also include initiatives aimed at making Digg more relevant to local tastes, including local languages. We will begin laying the groundwork for that immediately with the beginning of our international growth strategy in early 2009.

Other initiatives will focus on expanding our community outreach programs and developing more sophisticated tools and interfaces for publishers.

To help support all of this, we will be significantly growing the size of the Digg team in the next year and moving to larger San Francisco offices that give us the room we need to do it. We’re hiring, so check out our jobs page. Fueling the acceleration of all of these programs is our revenue success to date, along with a new $28.7 million round in funding, led by Highland Capital Partners.

Thanks again to all of you for your support in making Digg the vibrant community that it is today.

Digg on,
Jay

[Digg Company ]
View original post|Add to del.icio.us | Share

Hey Everyone -

A while back Katie Couric asked for some questions from the Digg community as she was heading out to the political conventions. You all suggested some great ones, and tonight she poses some Digg questions to both John McCain and Barack Obama. Tune in tonight to the CBS Evening News with Katie Couric to see if your question gets asked!

Digg on,
Kevin

[Digg Company ]
View original post|Add to del.icio.us | Share

Welcome back to the Digg technology blog, where you get to read about what the tech people at Digg are thinking about. Let’s get right into it, shall we?

How Many Databases does Digg Have?

As Joe hinted in his earlier blog entry, the particulars of how many machines Digg has is one of the most often asked questions, and yet one of the least relevant to Digg employees. None of us directly responsible for putting machines into our production cluster bothers to know the answer to this question, including myself.

The simple answer, which seems flippant, is that we have enough to do what Digg does. But I suppose you deserve a more detailed answer. So let’s go into it a little, shall we?

We have about 1.8x to 2.5x the theoretical minimum number of machines required to run Digg. Operations mandates that we want 2x the server capacity required to run Digg, so this makes sense. But why the spread? Let’s go into some of the reasons as they pertain to the databases.

Load deltas can be caused by errors or brilliance by Digg employees, such as bugs or fixes from the developers or improved indexing or bad I/O subsystem layout by the DB team, for example. Although it’s fair to say improvements far outnumber the mistakes, they both exist and both must be dealt with (feel free to ponder about what various ways performance improvements must be “dealt with”).

Sometimes Digg performance is changed when we enable or disable a particular costly feature of Digg. The DB team must be prepared for the human element of performance. Sometimes a feature is considered valuable enough to keep even though it causes what may appear to be an undue strain on the databases. At some time in the future, the feature may become so system-intensive that the hard decision is made to cut it. Until that point, the systems can be “overloaded,” and more importantly, once the feature is disabled, the systems become underutilised.

Pool utilisation can also be changed when the Digg members, as a social group, stop using a feature, or start using another that Digg itself made no change to.

All such factors contribute to the number of machines deployed in our production cluster at a given moment being either more or fewer than we desire, but if we’re careful, there are always more than the theoretical minimum required to run Digg, even during spikes in load.

Database Pools

At the highest level, you can think of the Digg databases as a four-master set of clusters. We shall call them A, B, C, and D. Two of the masters (masters A and B) are masters only, and two (masters C and D) are slaves of one of the other masters (A).

Each cluster has several slaves in it. I shall call a grouping of slaves a “pool.” At the highest level, you can think of all the slaves in a particular pool of masters B through D as equivalent, but the slaves in the pool underneath master A are special. This is for historical reasons. So let’s dig into history, shall we?

The original Digg database was designed as a single monolithic DB, and additional capacity was created by adding slaves. As Digg grew, we added more database clusters (B, C, D) into the mix. This is your classical scaling via distribution of writes, but the original database cluster (A) remained with most of the read/write throughput. Scaling it has proven to require a bit more ingenuity as its slaves have always historically had the most disk contention.

At first, to get more cache hits on a slave in A’s DB pool while still keeping all tables on each slave, they were split into subpools, call them A_alpha and A_beta. All queries sent to the slaves of A were given descriptors, and the database access layer was given a mapping of descriptors to subpools. Thus queries that mostly hit only tables M and N could be sent to A_alpha, and those that hit mostly I and J could be sent to A_beta. Hence the index and data pages for M, N, I, and J would most likely be in RAM on their respective database slaves.

This worked rather well for some time until the write load on A_[alpha|beta] became too intense, and further optimisations were required. These include dropping indexes and tables that aren’t needed in their respective subpools.

If you’re a MySQL DBA running MySQL 5.0 or lower, you know that there isn’t a simple report that MySQL will generate that shows you a list of indexes or tables that are used in a database. The assumption is that if your company created a table or index, that it will always be in use.

The radical changes to Digg’s front-end architecture over the past several years means that isn’t true. We know there are tables and indexes we don’t use in our A_alpha and A_beta subpools. So to determine what could be dropped from these two subpools, we analysed the dbmon.pl output (dbmon.pl is covered in some detail in the section on Database Overload, below) using getServerIndexes.pl. Basically it does an EXPLAIN on every query on the slave to generate a list of tables and indexes used (special note, I didn’t use the word “exhaustive” in the preceding sentence). If you use this tool, be sure you read the caveats in the script comments! As with everything high volume, nothing is ever simple.

Database Access Layer

The Digg database access layer is written in PHP and lives at the level of the application server (Apache). Basically, when the application decides it needs to do a read query, it passes off the query with a descriptor to a method that grabs a list of servers for the database pool that can satisfy the query, then picks one at random, submits the query, and returns the results to the calling method.

If the server picked won’t respond in a very small amount of time, the code moves on to the next server in the list. So if MySQL is down on a database machine in one of the pools, the end-user of Digg doesn’t notice. This code is extremely robust and well-tested. We worry neither that shutting down MySQL on a read slave in the Digg cluster, nor a failure in alerting on a DB slave that dies will cause site degradation.

Every few months we consider using a SQL proxy to do this database pooling, failover, and balancing, but it’s a tough sell since the code is simple and works extremely well. Furthermore, the load balancing function itself is spread across all our Apaches. Hence there is no “single point of failure” as there would be if we had a single SQL proxy.

Monitoring and Alerting

Though it is an integral part of our database architecture, I will keep this section a bit short, since it isn’t my specialty. We use Nagios to alert us on predicted failure modes of databases. The most common alerts are slave lag, disk space low, and complete machine death. Slave lag is caused by a number of things, including spikes in system usage, long-running update queries, or intermittent disk failure.

For monitoring, we use a Cacti-like Digg-written tool called MotiRTG. Suffice it to say it resembles Cacti in several ways, but is specialised to Digg’s cluster layout and is more suited to a cluster that has machines entering and leaving every day. It is written and maintained by our Networks and Metrics manager, Mike Newton. It is a strong candidate for open sourcing in the future.

Our alerting and monitoring subsystems are used in the traditional fashion. Alerts are for predicted failure modes, and monitoring is for post-failure analysis and future trending.

Database Overload

One of the most common problems on Digg systems is a spike in load, often caused by large news events like Apple announcements or hurricanes or… well, anything newsworthy. Assuming the spike isn’t taken care of by one of the myriad other spike-limiting features of Digg’s infrastructure, and the spike makes it to Digg’s databases, there are two simple mechanisms to limit the effect.

The first is the aforementioned over deployment of machines in our DB cluster. I estimate that this takes care of  more than 99% of database spikes in load. It may be no exaggeration to say 99.99%; we get several such spikes in database load per minute.

It is possible that a combination of adverse conditions will contribute to a spike that causes a particular segment of our DB resources to become 100% utilised. Under such conditions, it is not acceptable that Digg go down entirely. Hence the second mechanism, a tool called “dbmon.pl” (which you can download the source code for), is used. It’s a daemon that watches the MySQL instance for queries that have been running longer than a time limit set on the commandline, and kills queries that take longer than that.

Combined with separate DB subpools for different sorts of functionality on Digg, an overload on the DBs will only affect the portion of Digg serviced by that subpool, and then only a subset of the total requests coming in.

Note that the front-end application must deal with killed queries gracefully, since some of the killed requests will originate from legitimate users. If you use this tool, be sure you test your code in some environment where the kill time is set to a much lower value than you’ll actually use in production, and under stress, to be sure your application doesn’t barf out some nasty stack trace on the user when a connection gets killed.

Be very careful of adding logic where the front-end resubmits the request at no explicit request from the user. I recommend adding no such logic. It can easily negate the advantage of using dbmon.pl. Don’t worry. The user will hit reload. There is no need to DOS your own databases.

See You Next Time

Thanks for reading! I hope you found something useful or interesting here. This is Digg’s lead DBA Timeless signing out. Digg on.

[Digg Technology ]
View original post|Add to del.icio.us | Share

Hey all,

We’re growing up so fast…sort of. Today we launched two new official Digg Blogs: Community (this one) and Technology. We’ll still use our main blog for big news and other important announcements, but because we love you and you love us, we thought we’d add a few new sections to keep you in the know.

We leave you with a little glimpse of some of the people here at the Digg offices. We have crooning QA Managers, Product Managers with crabs and a little secret about what really powers the Digg Recommendation Engine. We are calling this video series ‘Safe for Work.’ Hope you like the first installment.

<object width="400" height="300"><param name="allowfullscreen" value="true"/><param name="allowscriptaccess" value="always"/><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=1706289&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/><embed src="http://vimeo.com/moogaloop.swf?clip_id=1706289&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>

Keep an eye on this space, more to come!

- Jen

[Digg Community ]
View original post|Add to del.icio.us | Share

We often get asked how Digg works from a technology perspective, so wanted to shed some insight on this with our first post from the newly-launched technology blog. We’ll be posting regularly to give you a peek at what’s under the hood from the Digg development teams.

Ask Ron - our Systems Engineering Lead - the exact number of servers we have in production and he’ll probably respond with, “I don’t honestly know.” I can say we’ve got dozens of web servers and dozens more DB servers. I can say with certainty it takes six specialized graph database servers to run the Recommendation Engine and we have another six to ten machines that serve files from MogileFS. But really, the numbers are the least interesting part of the equation. What makes Digg an interesting place to work are what the pieces are and how they fit together.

An average request to Digg starts at the outer most point in our architecture - the load balancers. We run a few specialized appliances that do a number of things: monitor the application servers, constantly adjust the cluster according to health, balance incoming requests and caching JavaScript, CSS and images. They’re also constantly keeping track of one another so they can take over all traffic if one of them fails. This is often referred to as a “heartbeat”. If you’re looking for a FOSS solution that offers a similar solution we recommend checking out the Linux Virtual Server and Squid.

Once your request has made it through the gauntlet that are the load balancers your request is handed off to an application server. Application servers consist of a number of daemons, including Apache+PHP, Memcached, and Gearman. These servers then work as conductors orchestrating DB connections, MogileFS connections and other service requests. It then munges, parses and processes the result and returns it to your browser.

The databases are broken up across four masters, each of which have lots of slaves dangling off of them. All writes go to the masters and all reads go to the various slaves. All run MySQL. Our databases are pretty vanilla as far as design goes though we probably have a lot more denormalized data than your average database design. Also on these machines are more Memcached and a specialized daemon that monitors connections and kills ones that have been open too long.

The file servers run Danga’s MogileFS, which is an anagram for “OMG Files!”. MogileFS is, essentially, a distributed WebDAV cluster. This serves up all of our story icons and our user icons. We also use it to store copies of each story’s source so Research & Development can study them.

That leaves us with the Recommendation Engine, which is a really fancy term for “distributed graph database”. Traditionally RDBMS’s are not well suited for doing the math that goes into generating a recommendation so Research & Development created a custom one.

So, that’s about it. Digg uses Debian GNU/Linux across the board with a mixture of MySQL, Memcached, MogileFS, Python, PHP, Apache, Gearman and various appliances to serve up billions of requests a month (and more every day!)

Thanks,

Joe

[Digg Technology ]
View original post|Add to del.icio.us | Share

Hey All,

A few folks have been discussing the use of scripts on Digg recently, so I wanted to jump into the conversations that happened this weekend. Scripts/bots place additional load on Digg servers (slowing things down for everyone), so using them is a Terms of Use violation that will result in losing access to your Digg account. We are currently looking deeper into recent script activity.

Digg monitors for script/bot activity globally across all our site pages. In addition to that, our development team has completed some improvements that will be rolled out this week. These changes will be more transparent by warning and preventing the users from using these scripts/bots. This will only affect a very small group of people, and the overwhelming majority of the Digg community won’t notice these changes. As always, all content on Digg is subjected to the Digg promotion algorithm, which requires a unique diverse pool of Diggers before promoting content to the homepage.

Please don’t hesitate to contact us at support at Digg with questions or feedback. We’re continuously researching and investigating these topics, so don’t be shy and let us know what you think.

Have a good one,
Jen

[Digg Website ]
View original post|Add to del.icio.us | Share

Hey Everyone -

We want to give the Digg community an opportunity to pose questions to some of the individuals and leaders of the moment (sans editors), who are taking action to change the world in cool ways. To do this, we’re launching a new program called “Digg Dialogg.” The concept is simple – we identify a featured guest that you will be able to submit questions to (text or video) which the Digg community Diggs up or down. We’ll pose the top questions to the guest during a live interview. Featured guests will represent thought leaders and tastemakers across diverse topics including technology luminaries, environmentalists, entrepreneurs, musicians and filmmakers.

We’ll be kicking off the program at the political party conventions, where we’ll be partnering with CNN’s iReport. Our first stop is the Democratic National Convention in Denver, where we will be hosting the interview on Wednesday 8/27. Our first guest will be Nancy Pelosi, Speaker of the House and one time recipient of the Digg effect. We will post the link to the live feed via the Digg Twitter account.

Go to the Digg Dialogg page or iReport to submit your questions and vote up or down the questions you’d like to see asked on the Dialogg thread.

We’ll be announcing more guests soon. Let us know who you’d like to talk to as we line up guests for the rest of the year by adding suggestions in the comments.

Digg On,
Kevin

[Digg Events Digg Website ]
View original post|Add to del.icio.us | Share

Hey everyone,

We’re calling for questions for our next virtual Townhall which will take place next Thursday, August 28th at 1:30 pm PST/4:30 EST.

This Townhall will be a bit of a twist, as it will be webcast live from the Digg Stage at The Big Tent at the Democratic National Convention in Colorado. (We’ll also be at the Republican National Convention the following week.) You can check out the Townhall webcast live here and it will be available for viewing anytime afterwards.

We’ll share out the latest Digg news and discuss the topics you’d like to see covered. We’re always looking for your feedback to help improve Digg, so let us know what you’d like to discuss by posting, Digging up, or burying the comments in this thread. We will also try to focus on new questions that we didn’t discuss in the last Townhall and omit any duplicate questions as we run down the list.

Looking forward to talking with you all next week!

- Kevin

PS: Don’t forget to subscribe to our Twitter account if you are in the Minneapolis/St. Paul area at the Republican Convention and are interested in attending our party.

[Digg Events Digg Website ]
View original post|Add to del.icio.us | Share

Hey all-

The politics section of Digg took off and has grown faster than almost any other content area on the site. Since then, we’ve added new categories for political opinion, political news and the 2008 U.S. elections. And, last fall we launched Digg the Candidates as another way to discover news on Digg and to follow content on the candidates and see what they are Digging.

As the U.S. election season hits full swing, we’re bringing this momentum to the upcoming political party conventions in Denver and St. Paul.

Next week at the Democratic National Convention, Digg, along with Google, local organizations, national blogs and others, will be sponsoring the first ever new media center at a major political party convention. The Big Tent will be a place for bloggers, new media journalists and the public to record their experiences from the convention and to hear from top newsmakers on the Digg Stage.

We’re also hosting our first live Townhall direct from The Big Tent on Thursday, August 28th at 4:30 EST/ 1:30 PST. We’ll be sending out a call for questions on the blog shortly.

The following week, we will be at the Republican National Convention in St. Paul. We’ve teamed up with MySpace, Rock the Vote and others to sponsor an event with the Impact Film Festival, which is celebrating the power of film to educate people on critical issues facing the world. We’ll be giving away a few free tickets, so be sure to subscribe to our Twitter feed for a chance to attend if you’re in town.

We’ll be taking videos and snapshots from both of the conventions and posting them on Digg, so you can follow all of the activity.

Thanks to all of you for your support in making Digg a great place for the best online content, whatever content you’re into.

Jay

[Digg Events ]
View original post|Add to del.icio.us | Share

Hey everyone –

Katie Couric’s asking for questions from the Digg community that she can take along with her to the political conventions in Denver and St. Paul. She’ll be interviewing newsmakers and politicians over the next couple of weeks and will post the videos on YouTube and CBSNews.com. If your question makes it, Katie will read your Digg username when she asks the question. To ask a question or vote on questions, visit her story here.

Digg on,

Kevin

[Digg Website ]
View original post|Add to del.icio.us | Share

Hey everyone -

We’ve just launched our first Firefox 3 extension and wanted to provide a quick video showing how it works. Some highlights include:

- Notifications: No matter where you are on the web, you can discover popular Digg stories and stay up to date on what your friends are Digging, submitting and commenting. These notifications are fully customizable. Within your preferences you can track different categories and media types, and turn notifications on or off.

- Digg Toolbar: Displays Digg counts & comments as you browse around the web. You can also Digg and submit stories directly from the toolbar, which is collapsible to save space. Note that the toolbar respects user privacy by passing only hashed URLs to Digg to check if they’ve already been submitted.

See it in action:

<object width="400" height="300"> <param name="allowfullscreen" value="true"/> <param name="allowscriptaccess" value="always"/> <param name="movie" value="http://www.vimeo.com/moogaloop.swf?clip_id=1444273&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/> <embed src="http://www.vimeo.com/moogaloop.swf?clip_id=1444273&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed></object>

It’s easy to install - click here and follow the prompts in Firefox.

If you haven’t yet upgraded to Firefox 3, Digg developer Kurt Wilms has put together a Firefox 2 extension that’s full of features - check it out here.

As always, we’re looking for your suggestions and feedback so please email them to us using our new contact form or comment below.

Digg on -
Kevin

[Digg Website ]
View original post|Add to del.icio.us | Share

Hey everyone -

It’s even easier to access Digg on the go with the new mobile version of Digg. This enhanced version - found at m.digg.com - is perfect for phones that support the full web browsing experience, such as a Treo, Blackberry or that new iPhone you just waited in line all morning to get.

This version provides:

- Multiple views of the most recently popular stories. Just as on Digg, you can tab between the filters to see stories popular in the last 24 hours, 7 days, 30 days and 365 days.
- Increased visibility to distinguish between media types. If the story is tagged as ‘YouTube’, you can click to play the video directly on your iPhone.
- The ability to favorite stories when logged in.
- Improved usability when surfing between various stories.
- The ability to load more comments on each story.
- Snappier page loads due to less javascript used on the site.

In case your phone doesn’t support a robust browsing experience, we continue to provide a basic mobile site at diggriver.com.

Enjoy!
- Jeff

[Digg Website ]
View original post|Add to del.icio.us | Share

Hi all,

It’s been a month since we’ve rolled out the Recommendation Engine, and we’ve found that you’re Digging more than ever. Now that we have thirty days under our belt, we wanted to share out some stats and give you a heads up on some of the improvements we are working on to make the Recommendation Engine even better. Some statistics:

- Digging activity is up significantly: the total number of Diggs increased 40% after launch.
- The Recommendation Engine is running strong: at any given point in time, the system is generating over 54 Million Recommendations, with the average Digger having nearly 200 Recommendations from an average of 34 “Diggers like you”.
- Friend activity/friends added is up 24%.
- Commenting is up 11% since launch.

Many of you completed the survey, thanks for all your feedback. We’re incorporating it into our next round of improvements, including:

- Adding more stories to the Recommendation Engine widget on the home page.
- Increasing the 30-day Digging window to a longer time frame, to get more Recommendations.
- Continued advances in the back-end algorithm to present you with more targeted Recommendations.

If you have feedback on the Recommendation Engine, we’d like to hear from you by filling out this quick survey or by adding your comments below. If you have yet to test drive the Recommendation Engine we encourage you to check it out. It’s a great way to discover relevant upcoming content on Digg.

Thanks everyone,
Anton

[Digg Website ]
View original post|Add to del.icio.us | Share

Hey Everyone—

Today we’re announcing more details around the new Facebook Connect feature. This feature will allow anyone who currently uses Facebook to seamlessly become a Digg user and start sharing and Digging stuff right away.

With this new feature, when you are prompted to log onto Digg, you can use your Facebook account information to sign in. Once you Digg a story, Facebook Connect can publish that in your Facebook Mini-Feed. This allows both Facebook and Digg users to more easily share the content they care about with the people they care about.

Facebook Connect is a key step in Digg’s effort to extend the Digg experience across the social graph. Look out for more upcoming features that further our commitment to the openness and transparency of user data and in providing you more control over your data.

Digg On,
Kevin

[Digg Website Upcoming Features ]
View original post|Add to del.icio.us | Share

Hey everyone -

The Digg crew is coming to Chicago this Wednesday. We’ll be at SmartBar in Wrigleyville from 6pm-10pm, so come out and say hi if you’re around.

It was great to meet some of you at the last New York meetup (one person even came from as far as Amsterdam to attend). We put together a quick video of some highlights from the last one:

<object width="400" height="300"> <param name="allowfullscreen" value="true"/> <param name="allowscriptaccess" value="always"/> <param name="movie" value="http://www.vimeo.com/moogaloop.swf?clip_id=1380549&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/> <embed src="http://www.vimeo.com/moogaloop.swf?clip_id=1380549&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300"></embed>

We hope to see some of you at our upcoming meetups. You can find more cities and updates here. Our next virtual Townhall will be in August.

Digg on,

Kevin

[Digg Events ]
View original post|Add to del.icio.us | Share

Hey Everyone —

We’re launching the Digg Recommendation Engine beginning this week. The feature will be in beta and presented to registered Digg users first, based on a random sampling of logged-in users. Look for the red beta flag on your Upcoming tab - this means you now have access to the Recommendation Engine. We hope to roll it out to everyone within the week or so.

The Recommendation Engine is a cool way to discover new content on Digg. Now that there are more than 16,000 stories submitted to the Upcoming section every day, it’s difficult to sort through everything to find the best content. The Recommendation Engine uses your past digging activity to identify what we call Diggers Like You (who you can see on the right hand nav) to suggest stories you might like.

You can see a video preview of Recommendation Engine here:

<object height="300" width="400">

<param name="allowfullscreen" value="true"></param> <param name="allowscriptaccess" value="always"></param> <param name="movie" value="http://www.vimeo.com/moogaloop.swf?clip_id=1233352&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"></param> <embed src="http://www.vimeo.com/moogaloop.swf?clip_id=1233352&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" height="300" width="400"></embed>
</object>

To understand more about how it works, you should check out our interview with Digg’s Lead Scientist Anton Kast:

<object width="400" height="300">

<param name="allowfullscreen" value="true"/> <param name="allowscriptaccess" value="always"/> <param name="movie" value="http://www.vimeo.com/moogaloop.swf?clip_id=1242909&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"/> <embed src="http://www.vimeo.com/moogaloop.swf?clip_id=1242909&amp;server=www.vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="400" height="300">
</embed>
</object>

You can also read more about the Recommendation Engine in Anton’s Whitepaper.

As you increase your activity on Digg, the Recommendation Engine will get to know you better and will suggest more targeted content and Diggers Like You. You can always remove specific users from the Recommendation Engine by clicking on the user name and selecting ‘remove from Recommendation Engine.’

Since this is the beta version we want to know what you think. Get back to us at feedback@digg.com with your thoughts and suggestions.

Digg on,

Kevin

[Digg Website Upcoming Features ]
View original post|Add to del.icio.us | Share

Hey Everyone —

We just released a new version of Digg the Candidates today - check it out if you’re interested in following the latest U.S. presidential election news on Digg.  The growth of the World & Business and Elections sections on Digg over the past year has been amazing, and it’s been great to see the diversity of interest from Digg users from all around the world.

We’ll be doing a few interesting things at this summer’s political party conventions. More details to come soon, but I wanted to give you a heads up there are a few projects the works.

Digg on,
Kevin

PS: We’ve also added a new topic for the Olympics.

[Digg Website ]
View original post|Add to del.icio.us | Share