Wednesday, January 13, 2010

Punishers and Reinforcers

OK- for anyone following this through the Bookends Farm fan page on Facebook, you already know I put up a link to an article by Karen Pryor about this topic today. Following that I was encouraged by EquiClick's Sarah Memmi to go ahead and post an equine version (thanks Sarah!). So here goes:

First, a quick definition of punishers and reinforcers:

Reinforcer- something which occurs during or immediately after an animal performs a behavior which increases the likelihood that the animal will repeat that behavior.
something which occurs during or immediately after an animal performs a behavior which decreases the likelihood that the animal will repeat that behavior.

So for anyone not familiar with the terminology, spend some time taking that apart. Forget your preconceived notions of punishment being something Bad or Mean. In Operant Conditioning language, in order for it qualify as punishment, it has to decrease the likelihood of that behavior happening in the future. So, for example, if you yell at a horse that is pawing on the
crossties and he stops pawing right then, but repeats the behavior 2 minutes later and the next time you go in the tack room and the next day, then it wasn't a punishment. Punishers don't have to work immediately, but they do have to decrease the likelihood so that using that punisher over time will cause that behavior will cease.

The other important thing to point out is that in order to be effective, punishers and reinforcers have to occur during or immediately after an animal performs a behavior. So shoving a dog's nose in a wet carpet when you return from work will not decrease the likelihood that he pees there again in the future. Oops, this was supposed to be equines. OK, so giving your horse a carrot when you get back to the barn after a good ride will not increase the likelihood that he performs well again tomorrow. It will increase the likelihood that he'll hustle back to the barn after your ride tomorrow because that's what happened right before he got the carrot! If you want to increase the likelihood of his doing a nice canter transition or jumping a ditch, give him a reward immediately. (And best of all, a click at the precise moment of his good behavior will be even more clear to him!)

Last of all, punishers and reinforcers are defined by the animal, not the trainer. So it doesn't matter whether you think you're reinforcing the animal or not, if the horse doesn't find what you are doing to be punishing or rewarding, then it won't affect his future behavior (or will affect it the opposite of the way you think. What does this mean? Some of my pet peeves:
  • riders who "pat" their horse on the side of the neck or, even worse, the top of the head, by pounding with the force that would knock over a small child. You frequently see this after a wonderful performance at a competition. The riders are ecstatic and they wave to the crowds as they pound on their horses. Looking closely, you can see the horses flinch! What horse likes to be pounded on, especially near the head?? A nice scritch on the withers will most likely be more appreciated. But will it change future performance? Debatable.
  • people who yell at their horses in the barn for banging on stall doors...and then throw them some hay to shut them up. Guess what? For most horses, a flake of hay is a lot more reinforcing than getting yelled at is punishing. They'll gladly bang on their stall again tomorrow, ignore the yelling and enjoy the following snack!
I'm not going to give anybody suggestions for good punishers because I prefer not to use them. But, for reinforcers, FOOD is great :) What kind of food? That depends on the animal AND the behavior being asked for. Your horse may be willing to work for hay stretcher pellets on a daily basis but if you're trying to work with trailer loading and a horse who has had an accident, you may need a more enticing treat to get him started...carrots, apples, peppermints, etc.

Rest can be another good reinforcer. If you are working a horse and he gives you a great moment of lightness or good transition, etc, drop your reins and give him a break. Many people want to push on at that moment because they've finally got it! But if it is hard for the horse, and you make him continue to work hard when he does something nice, well, he may think, ugh, I'm not doing that again- all it got me was more work! On the other hand, if you're working a thoroughbred or other horse who loves to Go, then by all means let him go on as a reward! So you need to know the individual you are working with to know what will work best for your horse. And it may be different on different days. Rest may be the best reinforcer in mid-summer when it's 90 degrees, whereas a nice trot would be more effective on a windy fall day.

One of the (many) enlightening things I remember hearing from Alexandra Kurland was her discussion of young horses who like to bite. Most handlers respond to biting babies with a swat- thinking it is punishing. But as Alex said, what most likely happens is that they think, oh great! Somebody to play with! They pull away and wait for another opening and take another swipe and quickly duck away before you can swat back. Ha- won that one! Just watch two babies together to see this game go on in equine form. My yearling and 2 year old will play this game on and off all day every day. They love it! Diving it to take a bite at a neck or nose or hock and then dodging away before the other one can retaliate. So, is swatting at a youngster punishing or reinforcing? I'd say reinforcing. To them it means "Game On!" And what is more effective? Well since they do it to initiate fun, if you walk away from them instead, then they don't get their fun. If they approach you politely and you offer to play games with them, then politeness will be reinforced. Think I'm nuts? I've followed this approach with all three of my youngsters since they were born and I've never been bitten. I've had plenty of swipes taken at me, but when I walk away they don't get the opportunity to do it again. When they approach nicely, I offer face rubs or clicker games or legitimate things to mouth (puppies are given chew toys, why not young horses?) That reinforces the pleasant approach.

Last but not least, it doesn't have to be the handler doling out the punishers or reinforcers. The environment can do it too. Think of a horse who slips on a trailer ramp, goes down and bangs his knees. You did nothing, but that horse will be less likely to go up that ramp confidently in the future. What about horses who figure out how to get out of fences...and get into deep green grass? I think I can say that good grass is about the most reinforcing thing in the world for horses. If they can repeat that breaking out behavior, they will! Going in a run-in shed to escape flies is an environmentally reinforced behavior. Pretty simple. (Running around like a madman in the pasture until someone brings you in is reinforced by the owner who goes tearing out to rescue the horse!)

So I have learned to think carefully about how I respond to the different things my horses and ponies do. I do try to focus on reinforcers and I observe to see how they respond to them. If they immediately repeat a behavior, I know I've got a good reinforcer!


Mary @ StaleCheerios said...

Great article!

I've got a bunch of nippy weanlings and yearlings that I need to do something about...
ignoring the behavior isn't working, but maybe walking away will. Attention is quite reinforcing.

It usually starts out so sweet though! Nice nuzzling and sniffing, and then someone will throw in a nip. Not hard, but hard enough that it needs to go away.


Bookends Farm said...

I have to say that one of the big advantages I have is that I am the only one who deals with my young horses 99.9% of the time. My husband will do chores for me on the rare occasion I can't, but otherwise, I am completely in control of the human feedback they get. So I think that makes a big difference. The lesson ponies, of course, are a completely different story- they are consistently reinforced for diving for grass by the young children who can't/won't stop them. Very frustrating!

One situation that has just "happened" which is a perfect setup for the youngsters is when they come looking for attention while I work in the barn. They hang their heads inside and I will stop what I'm doing and go give scratches or play head lowering games etc. The second that sweet nuzzling and exploring turns to a quick swipe (and so far it's been an "air grab"), I simply turn away and go back to cleaning stalls or scrubbing buckets etc. Game over. They get bored and leave. If they come back, I'll give them another chance as long as it's been a good period of time since the last try. If they just stand there, I usually ignore them for a good 5 minutes. But I can do this every it does sink in.
Did you ever have any success with the fearful horse you were working with?

Ark Lady said...

I did an article on punishment and reinforcers some time back (click my name to get there).

Soon I'll be moving everything onto the blog but it will forward just fine.

The topic confuses a lot of people because they assume what they identify as positive will be the same as the animal's--which you correctly point out is wrong.

Back to dogs, this is why so many jump and continue to jump. A push or eye contact is often enough to reinforce.

For the equine crowd Mary, I use a 60cc syringe to do a quick squirt (don't be overt) and usually that is enough.

Two things you can try:
-put the nuzzling etc., on cue so that it is reinforced and no reinforcement take place so you can control escalation.
-use something to redirect the bite to prior to it happening.

With predators I tend to use bite sticks but with other animals where nipping is part of their social regime, you need to redirect it.

Terminating an interaction might work but if you can distract the animal into something desirable, such as nipping translated to picking up something else with the teeth, you can help extinguish the problem.

Let me know how you tackle the problem.

Anonymous said...

One of my Chincoteagues is one of those wall banging horses when it comes to feeding time.

I recognized early on that dumping his grain in will certainly make him stop...but its reinforcing the behavior. So now he must stand quietly with all feet on the ground and ears forward. If he just can't control himself I feed everyone else but him and wait until he behaves properly to click and feed him.

The downside is that I guarantee that our farm helpers are most certainly not doing this. They are dumping the food just to quiet him.

I have noticed improvement and usually he needs a little extra time to realize...oh right its her, she won't feed me unless I stop. But unless I can control everyone else that comes in contact with him, there is no way I will ever fix this behavior. Can be very frustrating indeed.

Bookends Farm said...

Kyley, have you considered clicker training your farm helpers? :) You can do it "properly" with TAGteach or, as I suggested to a friend who had a problem with an apprentice who gave my friend's dog treats when she barked: give the apprentice chocolate when she reacted properly! Sometimes that game will just get them engaged enough to become interested in doing things right!

Anonymous said...

Love the article, very clear! I just came over from the Clicker Carnival, fun to find other positive trainers, even if for another species! :-)