Unless you can spend lots of time riding your horse, you probably should have him be able to roam around during the day in a pasture. The pasture must be fenced in with a strong, safe fence. It should be at least 4 feet high so your horse does not jump over it. It should be strong enough so that your horse can lean on it or rub its body on it.
The entrance gate must be hung so that it opens and shuts smoothly and firmly. It should be secured with a chain and lock. You should carry around a spare key to the lock. Horses are sneaky and might unlock it if there is no padlock on it. The gate should be wide enough so that you and your horse can fit through the gate side-by-side. The entrance might get a little muddy; to prevent this put sand around the gate.
A good fence should be strong and sturdy, and not have anything that might hurt the horse like sharp points that stick out. Strong wooden rails are good, but they cost more. Some thick rust proof wire strung through two posts are good and cost less. Make sure that the wire is one foot from the ground, so your horse will not get tangled in it. Thick hedges with no bad plants that might hurt horses are ok. They give a nice shaded place for horses on hot, sunny days.
Horses are much happier in a "herd" or group. When they are in a herd the bigger and stronger horses start picking on the little ones. They will push and nip at the smaller horses. To prevent this you should introduce the new comer, and watch it closely and make sure the new horse is not being bullied. You should also watch the youngest and oldest horses.