Your Cup Runneth Over:
Fill your head with information. The more intricate and related the details are the better. Watch Animal Planet, Discovery Channel. Go to the zoo. Read up on basic quadrupedal locomotion, how snakes move, what elephants eat.
This is a passive practice, you can do it anytime, anywhere. If something random catches your interest, allow yourself to follow it all the way through so you can learn all about it. This method is... 1. the secret to happiness, and B. the path to being a GREAT designer.
Lesson: Keep your eyes and ears open to information relating to animals (or ANYTHING) always. Got it?
A Strong House is Built on Solid Foundations
Consider your theme, genre, and style. Hyper realistic, fantasy, sci-fi, whatever it is? This is a great place to start because it will narrow your focus, and that is ultimately the purpose of any kind of concept design.
Removing all the possibilities until you come to the best design that fits the needs required by the art director. I find it's often easier to work inside at least some constraints than without any. When anything in the world is possible it can get a bit overwhelming to come up with something. So. For example, if you were working on a sci-fi project, you likely wouldn't create creatures that have weaknesses based on magic mirrors and such. The consumers of your work are getting more intelligent as a whole, and expect believability, so logic in relation to the rules of your world are paramount.
Lesson: Know what you are designing this creature for. If there is not much in parameters (like Conceptart.org's C.O.W.), create your own. Allow yourself to be strict with your work, as this can only strengthen your ability to deal with and create within the boundaries of the workplace project. This also forces you to be able to work in different styles. Which is the second most important asset to getting regular work. (The first is being easy and fun to work with. Most everyone would rather work with someone cool that's not the BEST in the world, than work with a brilliant yet fussy and unpleasant Prima Donna.)
The Answer to the Ultimate Question(s)
THINK! Think about your concept. Ask yourself questions to help bring it to life. Consider the whole biosphere. Not just the single creature you are working on and not just the Environment/ Atmosphere. Consider it's predators, it's prey, it's function in the 'chain of life'. If you don't know exactly what and how your creature eats, you've already lost the game. If you are going to ask yourself only ONE question, this better be it. Establishing where your critter sits in the food chain should be the first important fact you relate in your design. On a deeply animalistic level, I think humans need this kind of quick and easy visual assessment of animals in reality. They may not want to or be able to admit it, but somewhere in their medulla oblongata they are sizing up your creature wondering if they can club it to death and eat it, or if it will tear them to ribbons. Play with these subconscious concepts, and you will find your designs will be stronger than ever.
Please try and remember that there are more than just super predators out there. For example, the most common mistake I've seen in creature design is people essentially attempting to make a sloth into the Amazing Super Mega Conan X-men Ninja of sloths. Your creature does not NEED to be the destroyer of worlds to be cool. Unless the concept calls for it, of course. Understand that few if any creatures exist that have all the best features of all the best animals. Creature design, or any convincing storytelling related design has to also have weaknesses and short comings to feel real and relatable. Keep that in mind.
Lesson: Always be asking questions of yourself about your design. If you can't answer for why you made a certain anatomical decision, reconsider why it is there. This is where some artful bullshitting can be very useful, and the MORE you know about different animals the more you have to pull from to make a convincing reason. Do not misunderstand me. Being able to bullshit is not a negative thing, if done well it will only make your designs stronger and stronger. In a way, good design is just very intelligent and thought out bullshit. It's pretend! But, it can be damn good paying pretend, so... The More You Know... (shooting star)
Don't Always Reinvent the Wheel
Allow yourself at least at the beginning to heavily reference reality. Look at the plethora of dancing colors and bodies that are already all around us in the REAL world! Use big chunks of beasties that are already out there, the more recognizable, the more realistic it often will feel. Make sure that when you mix and match animal parts, you work out the new anatomy to accomodate your changes. If you design a caterpiller with human arms, make it feel natural, or it will scream "slapped together". And again, see the above section about being able to create good reasons for these choices.
If you want to break this thinking down into a formula to help get it into your head, then I suggest the following:
Make about sixty percent of your design recognizable referencing real animals. That sixty percent should mostly be one or two animals. Twenty percent should be lovely details or interesting features that you've found on animals that may not be as easily recognizable, and the remaining twenty can be COMPLETELY made up. This is just a rough idea to help get you going on the path to making realistic feeling creatures that feel like they fit into a real but unspoken biosphere. (Do not confuse this breakup of percentages with actual physical body mass. I am only speaking in terms of recognizability. If you've done it right people will observe, "Wow that looks like I would bump into that in the -insert random environment here-." Work on your percentages if they say something along the lines of: "That looks like a -blank- crossed with a -blank.")
This is just a quick and rough way to get a strong feeling of reality into your designs, and as you improve you will develop a much better system more fitting to you and your style, and that's just fine. Remember that nothing is written in stone.
-Lesson: Study the anatomy of everything. Learn the ins and outs of different animals to make logical decisions about which animals to mix, how, and why.
Drawing from Reality
Drawing FINALLY! Once you've picked a direction, find reference to all the animals that fit closely to your concept. If you are doing an amphibian kinda thing, look up frogs, salamanders, everything amphibian. Now DRAW. Draw the hell out of those things. Spend an hour or two just drawing real critters from photos, and be aware of a few things-
Shape and line: There are visual indicators that say frog, mouse, tapeworm, or goat. Learn the shape language for the animals you are referencing and be sure to use them later, as you sit down to work on the actual design.
Proportion: How can you suggest scale with your own designed creature? Learn the proportions of your reference animals so that tweaking them later will feel natural and still indicative of where they are coming from.
Coloring/ Patterning: Remember recognizable patterns and prints can suggest your reference, or adversely contrast the accepted pattern you would normally find. What does a diamond-back rattle snake look like if you use CANDY patterns instead?! It changes the whole meaning and attitude of the creature. Figure out why your ref. animals are colored the way they are. Knowing these different purposes can make your decisions for color and pattern easier and more logical when it comes to doing the final paints of your design.
Anatomy: Super important. Get to know the biokinetics of your creature's influences. This will help again with making things realistic. Having a good understanding of how different animals get around will expand the possibilities of your design's locomotion.
(Also... as you go along sketching/ researching or sketchearching (hehe) you will come up with tons of ideas on story details to make your creature more convincing... Mating habits, social interaction, communication... always keep that mind churning)
Lesson: This stage is vital, and I see it passed up all the time. Know your influences for a design intimately. This stage is perfect if you aren't an expert on your subject, and of course, each time you do this with something new, you add to your visual library as well as that all important bullshitting library. So don't skimp here. And, hell, do these kinds of animal and plant studies all the time, even when you aren't designing. Go to the Zoo and draw. Write little blurbs next to your sketches detailing the things you've drawn. Write the names of the animals, latin and common, these will always help you make more convincing critters later.
Closer to done than you think
This is where your different visual design methods will come into play. Some like to do high contrast thumbnails, some like to make loose pencil sketches till something comes out of it, other's like to find imagery in random coffee stains, or ceiling bumps. Whatever.
Just come up with a few different sketches to get an idea of your direction. This part should be easy and FAST if you've done all the above things, because believe it or not, in your head, you've been designing this whole time. Finding elements you like, and discarding ones that don't fit. When you have all the factual reality stuff from reference the rest is all up to you! Personally, I find I don't often do more than one or two sketches at this stage, because I've been developing the idea in my head in all those quiet moments of the day. Reading/ showering/ toilet time... whatever. Develop your own system. Remember your own system will ALWAYS work better for you, than trying to force yourself down another designer's flow. I don't mean for you to not LEARN different ways of doing it, that's plumb retarded; I am saying, once you know a variety of ways, go with your gut.
In essence, critter design can be likened to painting a room in a house. Most of the work should be done in the preparatory stage, getting things ready. If all the preparation is done correctly and carefully, the rest is a cake walk.
I hope this brief essay on critter design principles has been helpful, drop me an email anytime if you have questions, concerns, or suggestions (firstname.lastname@example.org)!
Remember to spay and neuter your monsters, beasties, creatures, and critters.
(I will be posting some sketches shortly, I had to publish this, to save it... the blog isn't saving shit for me. Bastard. Also, if you choose to link this, or use some of the text, please credit me by including a link to this blog!)