I think, like others have said, the biggest issue is genre. It's hard to dispute that Castle is well-made and well-written. However, it really does straddle that line between drama and comedy. The shows that win their respective categories also seem to be the ones that fit best into their categories, usually. 

Also, I do think that if anything, Castle would be put into the drama category, because the overall arcs of the show are drama. But when's the last time a network show has won drama categories? The last show to win best drama at the Emmy's that was on a network channel (as opposed to cable like HBO or AMC) was 24, in 2006. Right now, the drama categories are full of shows like Mad Men, Game of Thrones etc. hailing from cable, leaving little room for shows like Castle to get nominated.

Several shows live under the same gray umbrella with Castle, and, unless I'm mistaken, none of them got Emmy nods either.  The closest examples are CBS's "The Mentalist", TNT's "Rizzoli & Isles", A&E's "The Glades" and the (now defunct) "Crossing Jordan".  ABC's spinoff from Castle….. well, practically…. "Body of Proof" is yet another.  Watch these shows and you leave with a similar taste in your mouth.  A feeling of DejaVu.  (Wait, didn't I just see that actor playing a bad guy on Castle?  In a 3 season count, there were 32 character actors who played roles on both Castle, The Mentalist and The Glades.  Wait, Didn't Jane just say that on The Mentalist, as Castle says, "Because the heart wants what the heart wants.")  Castle is fun.  No doubt about it.  It's non-offensive with pretty people and ridiculously improbable scenarios, much the same as the aforementioned, so my guess is there's nothing to really grab the Emmy voter as some standout piece of writing or acting that raises the hair on the back of your neck or wrenches your gut with such emotion that you can feel the pleasure or the pain.  There's no soliloquy from Martin Sheen as he rages against the televangelist or the tears shed by CJ as her Secret Service body guard lies dead in a New Your City convenience store.  There's no writing like that delivered by Rob Lowe as spoke about the heroes of the assassination attack.  This is no West Wing.  This is no NYPD Blue.  That being said, neither is it "Lost" or "fill-in-the-blank-reality show" that seems to permeate our television sets.  At what point did someone decide a "reality" show has merit in any category for any purpose?  Does the audience not know there are camera crews standing 5 feet away taking these images?  Do they not know there are lunch and dinner breaks and no-one is actually going to starve or die?   And does anyone really, really, really care what's going on with the Khardashians or the dysfunctional Teutels?

Where did "entertainment" come to mean we have to be subjected to the likes of Kat Von whatever and Snooki?  Sadly, as long as shows like these are the yardsticks by which entertainment is determined, shows like Castle, will continue to be passed over at awards time.

Being sorry is a far worse punishment than being dead. Everybody dies, but very few people ever feel truly sorry for the bad things they've done.