You can normally get a good horse for about $3000. However, the cost isn't the purchase price, but rather how much it costs to keep the horse. If you self-livery (stable him on your property and do the stable work yourself) a typical cost is $200/month. If you use a professional stable, it will be double or three times this.
There are many factors that will determine how much your horse will cost to keep: - Stable. Professional or self-stable or pasture - Location. Cost of stabling, bedding, feed and other items vary widely from one area to another. You need to phone around to get your local costs. - Use. Using your horse (riding, exhibitions, competition) incurs additional costs.
A list of the various costs, along with an online calculator is provided at http://www.wowhorses.com/cost-of-a-horse.html .
I would suggest you go through the list and see what is applicable to you. Then phone around for local costs and plug them into the calculator. Finally, add 25% for items you missed or contingencies. Should give an approximate number for your objectives and circumstances.