SEO's in the past used several different techniques to artificially inflate rankings of their clients with 1000's of backlinks, or spammy web pages solely designed for seo. Google decided to take action against these type of tactics by implementing penalties on websites. The most common algorithm updates that lead to Google penalties are Google Panda and Google Penguin updates. Recently Google also did an update nickname Pigeon that effected the Google Local Maps placements.
A penalty can manual or algorithmic. A manual penalty is where someone from within Google has manually reviewed your site and determined that it violates one or more of their guidelines. An algorithmic penalty means your website is not in compliance their new update and will cause your rankings to fall.
Both of these types of penalties require work to fix or remove. A manual action means you will need to actually contact google with the progress you have made to remedy the situation. This means submitted an email to Google with the work you have done and asking for the penalty to be lifted. This process can take months and be pretty expensive. An algorithmic penalty means you should use link removal to clean up your backlink profile.
If you do get a manual action, it will be listed in your Google Webmaster tools page. If you do not have your site in webmaster tools, you should get this set up immediately.
The Penguin update focused on keyword backlinks, or poor back links. If your site was #1 of google for a term because you had 1000's of links pointing back to your website with that term, you probably have a penalty.
The panda updated mainly went after websites with a lot of low quality pages that were mainly created to influence search engine rankings. If you site has several pages under 100 words, stuff with keywords, you are at high risk for a Panda Penalty. It would be a good idea to get rid of these pages and replace them with high-quality content with at least 300-500 words each.
If you are concerned you might have a Google Penalty, give us a call and we can help you find out.