Clicker Training and Positive Reinforcement methods are in an interesting place right now and I have to be honest and say it's a little scary. Its successes are attracting attention and getting press. There has been a 2 series article in "Practical Horseman" magazine recently as well as mention of it in a feature article on the jumper stallion Judgment in "The Chronicle of the Horse". So it's getting out there to the masses which is a good thing.
But along with the popularity comes a lot of misunderstanding and that's what worries me. People try to simplify it into an easy recipe: "just reward your horse when he does something right". Yeah, well, sorta. Or they think pressure and release is positive reinforcement (it's not- it's negative reinforcement). Or they feed their horses sugar cubes, thinking they are doing some good when their timing is such that if the horse learns anything from it, it's not what the people intended. Or they refuse to "bribe" their horses with food (clicker training is not bribery).
When it comes to teaching other humans, people fall into the trap of mindless praise. They think positive reinforcement means not screaming at students, but instead gushing over them and telling them "good job" every time they cross the street without falling down. That just creates "praise junkies"- people who can't function without begin told how wonderful they are all the time, and that is pretty damaging to kids. In the world of TAGteach, we use positive reinforcement along with an acoustical marker and other specific techniques. TAG teachers are certified after training and hours of documented and approved practice.
My current feeling is that to understand Clicker Training and positive reinforcement, you really have to have a basic understanding of Operant Conditioning. But some people just glaze over when you try to explain that. They don't want the gory details. They just want the magic that is in the little clicker box and well, sorry, no magic involved.
The whole thing makes me occasionally want to bang my head against a wall because the very notion of positive reinforcement does not allow me to shake these people and scream "You don't get it!". My biggest fear is for all those who try this watered down, incorrect and/or misunderstood version and walk away from it because it didn't "work". It works. But you have to do more than scratch the surface. And for those who do, we are rewarded with some of the most amazing eye-opening experiences and relationships you could ask for.
So I shall try to refrain from banging my head and shaking people. Instead I will remember the tenet of "babysteps". I will try to teach one person at a time, in a way that they find reinforcing and hopefully thereby help one horse at a time. And I will be so appreciative of those who are willing to educate themselves rather than look for someone to hand them a magic potion which will ensure rapid success.
Thursday, December 17, 2009
Monday, December 14, 2009
Another thought on shut down
As I was mucking stalls this morning, I was thinking about horses in my past and the many horses you see in the stabling area of competitions who are rock steady in their manners on the ground. They will stand while they are groomed, wrapped, braided, etc, frequently without even being tied. But for many of these horses, it's because they have learned it's better to do nothing until told to do something. So they stand still unless they are pulled (rope) or pushed (shoved over). Rather than being taught TO stand, they have been taught NOT to move. There is a difference.
The first difference is the way that it is taught- with reinforcement or with punishment. Clicker training horses to stand still is done with reinforcement- stand still and you get a treat, now stand still longer and get another treat, etc. They learn self control leads to rewards. They learn good things happen if they stand still. Horses who are taught NOT to move are usually taught with punishment. If they move, they get yelled at, slapped, shanked, yelled at some more, etc. Pretty soon they are afraid to move because if they do, something bad will happen...that's fear.
The second difference is what will make these horses move and what happens when they do move for some reason. The horse who has been taught not to move by using punishment will stand until he becomes more afraid of something else than he is of the punishment. It may be a door banging, a dog barking down the aisle, a loose horse outside, kids running and screaming etc. The same things might cause a positively reinforced horse to startle but it is more likely that his frame of mind is calmer. And if he does startle, the response will be different. Rather than over-riding fear, he has been conditioned to think about the good things that will happen if he stands. So rather than spooking and then taking off for fear of the retribution, he will spook and then think, "oops, if I stand or go back to my person, good things will happen". A horse standing out of fear may be on edge on a windy day because he is nervous but is afraid to move for fear of punishment so the tension builds and builds in his body until he busts out with some nervous energy or becomes so afraid that he decides it's better to leave town than stay put. A horse standing from being rewarded will have a calmer, happier frame of mind and will be conditioned to calm himself in order to get rewarded. In addition, his person will most likely see his nervousness and begin reinforcing him more for standing as well as asking him to do relaxing things like dropping his head which is a conditioned activity for calming. That horse will feel safest with his person, doing what he knows will be reinforced. The other horse may very well decide that he needs to take matters into his own hands since his handler is scary and so is the banging door.
In the end, I'd rather have the reinforced horse under me when riding than the punished horse. I feel safer on a horse who looks to me for help in calming down than the one who may decide that truck coming down the road is scarier than the thought of any punishment I would dole out, especially if he dumps me just to avoid both options.
The first difference is the way that it is taught- with reinforcement or with punishment. Clicker training horses to stand still is done with reinforcement- stand still and you get a treat, now stand still longer and get another treat, etc. They learn self control leads to rewards. They learn good things happen if they stand still. Horses who are taught NOT to move are usually taught with punishment. If they move, they get yelled at, slapped, shanked, yelled at some more, etc. Pretty soon they are afraid to move because if they do, something bad will happen...that's fear.
The second difference is what will make these horses move and what happens when they do move for some reason. The horse who has been taught not to move by using punishment will stand until he becomes more afraid of something else than he is of the punishment. It may be a door banging, a dog barking down the aisle, a loose horse outside, kids running and screaming etc. The same things might cause a positively reinforced horse to startle but it is more likely that his frame of mind is calmer. And if he does startle, the response will be different. Rather than over-riding fear, he has been conditioned to think about the good things that will happen if he stands. So rather than spooking and then taking off for fear of the retribution, he will spook and then think, "oops, if I stand or go back to my person, good things will happen". A horse standing out of fear may be on edge on a windy day because he is nervous but is afraid to move for fear of punishment so the tension builds and builds in his body until he busts out with some nervous energy or becomes so afraid that he decides it's better to leave town than stay put. A horse standing from being rewarded will have a calmer, happier frame of mind and will be conditioned to calm himself in order to get rewarded. In addition, his person will most likely see his nervousness and begin reinforcing him more for standing as well as asking him to do relaxing things like dropping his head which is a conditioned activity for calming. That horse will feel safest with his person, doing what he knows will be reinforced. The other horse may very well decide that he needs to take matters into his own hands since his handler is scary and so is the banging door.
In the end, I'd rather have the reinforced horse under me when riding than the punished horse. I feel safer on a horse who looks to me for help in calming down than the one who may decide that truck coming down the road is scarier than the thought of any punishment I would dole out, especially if he dumps me just to avoid both options.
Monday, December 7, 2009
Loopy Training
Alexandra Kurland presented us with a new term this year- loopy training. Katie Bartlett does a wonderful job (as always) in explaining it at her website www.equineclickertraining.com To really get a good idea of what loopy training is, you should read her description but to put it very simply, (and looking through my understanding) it's looking at the flow of your and your horse's actions when you ask for a specific behavior. A very simple example is asking your horse to target something. The loop begins when you give the cue (hold out the target), continues as your horse touches the target, then you reward and the loop is complete when you ask for the behavior again. You can then go into another loop (either the same behavior or another one). The significance of this is how "clean" your loops are. A clean loop is one that does not contain any other other behaviors than the one you ask for. So this would require that your horse promptly touch the target when it's presented, that you click and deliver the reward promptly, that the horse have established manners for collecting his reward and at the end you are in a place where you can ask for the behavior again. A cluttered loop could include the horse being distracted by something when you hold out the target so that he doesn't immediately target it, could include you fumbling for the treat in your pocket, could include the horse being rude by reaching or stepping into your space for the treat, etc. You want clean loops for clarity of training. All that other stuff in the loop gets built into the final behavior if we, as trainers, don't clean it up.
To project this into the riding sphere, you could examine your canter transition. A clean loop would begin with your aids for the canter and would be followed by the horse jumping promptly into a balanced canter, followed by a reward (whether it is a click and treat, a kind word or scratch, or in the case of negative reinforcement, simply the prompt removal of the aids) all finishing with the horse being ready for the next request. A cluttered loop could include unclear or incorrect use of aids, aids timed incorrectly, uncoordinated use of aids including excess movement of the rider, all of which could result in a horse who may not take the aid, might rush into the canter, might show unhappy signs such as tail wringing or ear pinning, might not transition into a balanced canter and/or might canter off but would not be immediately able or willing to accept the next aids from the rider.
It's been fascinating for me to observe my own training in this context. I have been trying to clear up a lot of sloppy handling on my part. Clicker training has certainly made me aware of how I was always "shouting" at horses, not literally, but with my aids, and how unnecessary that is. Considering that my yearling, Percy, will walk alongside me, without halter or lead, politely and at an acceptable distance, and he will halt, trot or step laterally when I do the same, I really have no need for shouting of any kind. He does this as I walk through his paddock to get to the barn and I do not click or treat for any of it unless he does something fantastic...it's just the way we proceed from fence to barn these days. But it does mean that I am very careful of my thoughts and movements as he accompanies me. If my mind is elsewhere, my body is not projecting what I want from him. I may be hurrying to get chores done....and he reflects that back by hurrying and perhaps getting a little wound up. So the loop when I enter his paddock has become: duck under the fence, slowly stand upright because he is right there hovering, plant the expected kiss upon his muzzle as he sniffs my face (bad, bad horse trainer), step off toward the barn quietly but with intention and proceed with him at my side. There are many bits of this loop and all have been built individually before being added together, but now I simply have to keep myself attentive so that this loop stays clean- distractions pop up and that is all part of the training process. I need to make sure that I respond in a way that does not reinforce unwanted bits.
To go back to my recent Ear Obsession post, Ande putting his ears back was a piece of clutter that was built into several loops. I wrote about how I used going toward grass as a reward for him keeping his attention on me and with ears relaxed. I also intentionally built some loops once we got to the grass. When we first got to the field, I would let him graze until he was a little less frantic- initially tearing off hunks of grass and then settling into a grazing rhythm. I wanted to set us up for success. Then I would ask him to lift his head for a moment by just putting a little pressure on the lead. As soon as he did, I said "graze" and immediately let him go back to eating. I let him graze for another minute or two, then repeated but asked him to hold his head up for just a second or two. If he tried to put his head down, I simply anchored my hand to my hip so he couldn't. When he was still for a second or two, then I said "graze" and bent slightly at the waist to invite him back to the grass. It was amazing how quickly he picked up the graze cue. The cue "graze" became a reinforcer. I was not clicking because he would have looked to me for a treat- I wanted the action of grazing itself to be the reward.
At this point the loop was- cue for head up, he lifts his head promptly from a light rope cue, waits quietly, I give the cue "graze" and he puts his head back down and is ready for me to ask again. I began to lengthen my loop, adding in a couple walk steps: rope cue to lift head, cue to walk off, walk a step or two, say "graze" and let him go back to eating. What we were practicing was his response to my request for attention even when he is engrossed in heavenly deep fall grass, his willingness to leave the grass, and his waiting for a cue from me before diving for the grass, rather than just pulling for it all the time. Over the weeks of November when we were so lucky to have such nice weather this year, we continued with this exercise a couple times a week and built in walking down the road. He was allowed to graze for a bit, then I asked him to leave the grass, walk down the road with me a ways (slowly building up distance so that I returned to the grass BEFORE he showed any nervousness...I did not want to turn around and reward him for showing signs of anxiety) and then we'd return to the grass but he could only graze when he heard the cue. I'm glad I did that in November because as of this week, this is what that field looks like now!
To project this into the riding sphere, you could examine your canter transition. A clean loop would begin with your aids for the canter and would be followed by the horse jumping promptly into a balanced canter, followed by a reward (whether it is a click and treat, a kind word or scratch, or in the case of negative reinforcement, simply the prompt removal of the aids) all finishing with the horse being ready for the next request. A cluttered loop could include unclear or incorrect use of aids, aids timed incorrectly, uncoordinated use of aids including excess movement of the rider, all of which could result in a horse who may not take the aid, might rush into the canter, might show unhappy signs such as tail wringing or ear pinning, might not transition into a balanced canter and/or might canter off but would not be immediately able or willing to accept the next aids from the rider.
It's been fascinating for me to observe my own training in this context. I have been trying to clear up a lot of sloppy handling on my part. Clicker training has certainly made me aware of how I was always "shouting" at horses, not literally, but with my aids, and how unnecessary that is. Considering that my yearling, Percy, will walk alongside me, without halter or lead, politely and at an acceptable distance, and he will halt, trot or step laterally when I do the same, I really have no need for shouting of any kind. He does this as I walk through his paddock to get to the barn and I do not click or treat for any of it unless he does something fantastic...it's just the way we proceed from fence to barn these days. But it does mean that I am very careful of my thoughts and movements as he accompanies me. If my mind is elsewhere, my body is not projecting what I want from him. I may be hurrying to get chores done....and he reflects that back by hurrying and perhaps getting a little wound up. So the loop when I enter his paddock has become: duck under the fence, slowly stand upright because he is right there hovering, plant the expected kiss upon his muzzle as he sniffs my face (bad, bad horse trainer), step off toward the barn quietly but with intention and proceed with him at my side. There are many bits of this loop and all have been built individually before being added together, but now I simply have to keep myself attentive so that this loop stays clean- distractions pop up and that is all part of the training process. I need to make sure that I respond in a way that does not reinforce unwanted bits.
To go back to my recent Ear Obsession post, Ande putting his ears back was a piece of clutter that was built into several loops. I wrote about how I used going toward grass as a reward for him keeping his attention on me and with ears relaxed. I also intentionally built some loops once we got to the grass. When we first got to the field, I would let him graze until he was a little less frantic- initially tearing off hunks of grass and then settling into a grazing rhythm. I wanted to set us up for success. Then I would ask him to lift his head for a moment by just putting a little pressure on the lead. As soon as he did, I said "graze" and immediately let him go back to eating. I let him graze for another minute or two, then repeated but asked him to hold his head up for just a second or two. If he tried to put his head down, I simply anchored my hand to my hip so he couldn't. When he was still for a second or two, then I said "graze" and bent slightly at the waist to invite him back to the grass. It was amazing how quickly he picked up the graze cue. The cue "graze" became a reinforcer. I was not clicking because he would have looked to me for a treat- I wanted the action of grazing itself to be the reward.
At this point the loop was- cue for head up, he lifts his head promptly from a light rope cue, waits quietly, I give the cue "graze" and he puts his head back down and is ready for me to ask again. I began to lengthen my loop, adding in a couple walk steps: rope cue to lift head, cue to walk off, walk a step or two, say "graze" and let him go back to eating. What we were practicing was his response to my request for attention even when he is engrossed in heavenly deep fall grass, his willingness to leave the grass, and his waiting for a cue from me before diving for the grass, rather than just pulling for it all the time. Over the weeks of November when we were so lucky to have such nice weather this year, we continued with this exercise a couple times a week and built in walking down the road. He was allowed to graze for a bit, then I asked him to leave the grass, walk down the road with me a ways (slowly building up distance so that I returned to the grass BEFORE he showed any nervousness...I did not want to turn around and reward him for showing signs of anxiety) and then we'd return to the grass but he could only graze when he heard the cue. I'm glad I did that in November because as of this week, this is what that field looks like now!
Saturday, December 5, 2009
Ear Obsession
I'm becoming really obsessed by the position of horses' ears. I do not like seeing horses with their ears back when they are worked, whether in hand or under saddle because to me it signifies unhappiness. But the more I watch horses, the more I wonder about it all. For instance when you watch horses in the circus or Lipizzaners, they have their ears back. Perhaps it's the way they have been trained- I always assumed whips and things would bring out an angry attitude. But watch horses gallop in the pasture and they have their ears back- even if they are free and alone. Is it sometimes a sign of concentration? Is that why so many clicker trained horses frequently have their ears back while being worked in hand? Certainly sometimes it is handler error, but it has been interesting to think about and study. Here is a photo of Ande just after I have asked him for a trot transition on the longe. I have been clicking him for a prompt little bounce into the trot, rather than shuffling into it. That seems to take more effort- and the ears went back:
Ande is the one with which I have the most difficulty with his ears and I've written about it before. He is much better than he used to be and I have learned a lot from working with him on it. One of the most important things I finally figured out was how many times he was pinning his ears at the other horses outside the round pen. I figured this out by clicking him for duration with an ear up. I started by just waiting for him to put one ear up at a standstill and clicking that. I did not go for two ears, because my understanding is that a horse puts two ears up when he is using his binocular vision as opposed to his monocular vision. So as long as his two eyes are focused on two different things (which we as humans can't do), then his ears are pointed different ways as well. But when something is so interesting- either appealing like a bucket of grain rattling or scary like something in the bushes, then he focuses both eyes on that spot and both ears go up. I was not interested in creating a picture-pretty pose, although you can see from these photos that he is much cuter when his ears are up! I was interested in assessing his attitude. I think when he was younger, I unknowingly reinforced him for putting his ears back by not paying attention to what his ears were doing when I worked with him a lot. That's why I've put so much energy into trying to untrain that now. My concern was that he was feeling grumpy at one point, put his ears back, got reinforced, and it told him, "ah, if I threaten her, then she'll click and treat me". But really, there was absolutely no other aspect of his behavior that felt threatening to me. He never tried to bite, was very polite when taking treats, and as a matter of fact, when he hears the click, his ears swivel forward in anticipation. I would think if he was threatening, he would keep the ears back to tell me to hurry up and feed him. Not so. Here you can see his attitude as I coil up the longe line- curious and polite:
So back to the other horses- as we worked on duration when walking forward, I found there were certain sections of the round pen where he would happily leave his one ear forward for 10 or more steps, but we had trouble getting beyond that...and that he had a hard time putting his ear up if we were in other parts of the pen. That is when I noticed that it was when we were across from other horses that he put his ears back, and in the parts of the pen when there was just open field beyond, he was happy to keep an ear up. So now I think he was in fact threatening the other horses, telling them, "this is my time and she's my person right now- you guys stay away!" So that was actually a good thing- he wanted to be working with me.
Another technique I used with him was the Premack principle which states that more probable behaviors will reinforce less probable behaviors. Rather than clicking him for ears up, I used another behavior he wanted to do as reinforcement. In this instance, I had wonderful fall weather and hayfields which were open and done producing for the season so I was allowed to hand graze in them. I took Ande out there once or twice until he knew where we were headed when I took him out and he was anxious to get to the field. The grass in the horse paddocks was long gone so he hadn't had grass in a while and it was very exciting! My method was to stand with him until his ears (or at least one) went up, and then allow him to walk forward...toward the grass. If his ears went back, then I stopped (a little negative punishment there). As soon as his ears went up, then we walked forward again. He learned that he had to have a pleasant expression in order to be allowed to proceed toward the grass.
At this point, I was also careful to observe his overall attitude as well as his ears. Last winter I made the mistake of clicking him for ears up when something in the distance attracted his attention. My thought was that I didn't care what caused it, I wanted to give him the idea that his ears should be up. The problem was that this reinforced him for looking off in the distance rather than paying attention to me, which I think exacerbated the bolting problem I was having at the time....look away, find something scary to focus on, then take off for the barn. Not good. So with the grass, I actually preferred only one ear up and attention on me. It was pretty funny the first time because he tried all sorts of things and finally looked at me as if to say, "what the heck do you WANT?" and that's when I let him go forward. So it quite nicely evolved into a process where when he wanted something- to go to the grass- he would look to me- perfect. I want a horse checking in with me mentally to see what comes next and if he wants something, to look to me for help getting it rather than thinking about getting away from me to get what he wants.
As a result of all this, I have become much more in tune with the attitudes I am working with. I pay attention to lots of other signs besides ears: eyes, expression, breathing, etc. It helps me know whether they are tense and "being good" in order to get a treat, or if they are truly relaxed and attentive. Thanks to Lindsey for her recent question on my first ears post which inspired me to revisit this!
And here is the little Ande man, ears up and getting rewarded for it! If anybody knows a small adult or capable child looking for a nice coming 4 year old pony.....
Ande is the one with which I have the most difficulty with his ears and I've written about it before. He is much better than he used to be and I have learned a lot from working with him on it. One of the most important things I finally figured out was how many times he was pinning his ears at the other horses outside the round pen. I figured this out by clicking him for duration with an ear up. I started by just waiting for him to put one ear up at a standstill and clicking that. I did not go for two ears, because my understanding is that a horse puts two ears up when he is using his binocular vision as opposed to his monocular vision. So as long as his two eyes are focused on two different things (which we as humans can't do), then his ears are pointed different ways as well. But when something is so interesting- either appealing like a bucket of grain rattling or scary like something in the bushes, then he focuses both eyes on that spot and both ears go up. I was not interested in creating a picture-pretty pose, although you can see from these photos that he is much cuter when his ears are up! I was interested in assessing his attitude. I think when he was younger, I unknowingly reinforced him for putting his ears back by not paying attention to what his ears were doing when I worked with him a lot. That's why I've put so much energy into trying to untrain that now. My concern was that he was feeling grumpy at one point, put his ears back, got reinforced, and it told him, "ah, if I threaten her, then she'll click and treat me". But really, there was absolutely no other aspect of his behavior that felt threatening to me. He never tried to bite, was very polite when taking treats, and as a matter of fact, when he hears the click, his ears swivel forward in anticipation. I would think if he was threatening, he would keep the ears back to tell me to hurry up and feed him. Not so. Here you can see his attitude as I coil up the longe line- curious and polite:
So back to the other horses- as we worked on duration when walking forward, I found there were certain sections of the round pen where he would happily leave his one ear forward for 10 or more steps, but we had trouble getting beyond that...and that he had a hard time putting his ear up if we were in other parts of the pen. That is when I noticed that it was when we were across from other horses that he put his ears back, and in the parts of the pen when there was just open field beyond, he was happy to keep an ear up. So now I think he was in fact threatening the other horses, telling them, "this is my time and she's my person right now- you guys stay away!" So that was actually a good thing- he wanted to be working with me.
Another technique I used with him was the Premack principle which states that more probable behaviors will reinforce less probable behaviors. Rather than clicking him for ears up, I used another behavior he wanted to do as reinforcement. In this instance, I had wonderful fall weather and hayfields which were open and done producing for the season so I was allowed to hand graze in them. I took Ande out there once or twice until he knew where we were headed when I took him out and he was anxious to get to the field. The grass in the horse paddocks was long gone so he hadn't had grass in a while and it was very exciting! My method was to stand with him until his ears (or at least one) went up, and then allow him to walk forward...toward the grass. If his ears went back, then I stopped (a little negative punishment there). As soon as his ears went up, then we walked forward again. He learned that he had to have a pleasant expression in order to be allowed to proceed toward the grass.
At this point, I was also careful to observe his overall attitude as well as his ears. Last winter I made the mistake of clicking him for ears up when something in the distance attracted his attention. My thought was that I didn't care what caused it, I wanted to give him the idea that his ears should be up. The problem was that this reinforced him for looking off in the distance rather than paying attention to me, which I think exacerbated the bolting problem I was having at the time....look away, find something scary to focus on, then take off for the barn. Not good. So with the grass, I actually preferred only one ear up and attention on me. It was pretty funny the first time because he tried all sorts of things and finally looked at me as if to say, "what the heck do you WANT?" and that's when I let him go forward. So it quite nicely evolved into a process where when he wanted something- to go to the grass- he would look to me- perfect. I want a horse checking in with me mentally to see what comes next and if he wants something, to look to me for help getting it rather than thinking about getting away from me to get what he wants.
As a result of all this, I have become much more in tune with the attitudes I am working with. I pay attention to lots of other signs besides ears: eyes, expression, breathing, etc. It helps me know whether they are tense and "being good" in order to get a treat, or if they are truly relaxed and attentive. Thanks to Lindsey for her recent question on my first ears post which inspired me to revisit this!
And here is the little Ande man, ears up and getting rewarded for it! If anybody knows a small adult or capable child looking for a nice coming 4 year old pony.....
