Friday, November 21, 2014

When Can I Stop Clicking and Feeding? Building Chains and Sequences

A quiet and attentive Percy
I think everyone who has ever looked into Clicker Training has asked the question, "When can I stop clicking and treating?"  I know as a trainer, I have heard it more times than I can count and I also know I used to ask that same question myself.  I have heard, and given, many responses. Last week I had the opportunity to ask Percy that question. One of Alexandra Kurland's expressions is, "Go to people for opinions and horses for answers".  

Percy has learned everything in life with clicker training.  He learned how to lead, how to stand for grooming, how to get on a trailer, how to wear a saddle, how to take a bit- everything has been trained with clicker training.  Furthermore, everything has been maintained, at times, with the clicker.  I did not train him to stay by my side with a lead on and then stop using the clicker and treats for leading.  I know that behavior which is reinforced is more likely to be repeated.  I knew that if I was leading him out to the pasture, he was being reinforced by being turned out- no click and treat necessary for going forward with me.  There was, however, the occasional need for walking politely next to me rather than pulling ahead to get turned loose sooner.  If we are going somewhere new, or close to something which is scary looking, I click and treat steps forward- that's a more difficult behavior than walking with me to the pasture. I continue to refine all behaviors and that's one reason to keep clicking and treating.

I knew that by starting all grooming experiences with a clicker and treats had turned him into a horse who enjoys being groomed and so lots of times I groom him from forelock to hoof with no clicks or treats. I make sure the tools I use and the way I use them are reinforcing and so I do not need anything else.  On those days, the experience of being groomed even reinforces standing still on the cross ties.

Other days it is windy and there are noises which concern him.  Those days I click and reinforce head height as that is an indicator of his comfort level and I want him to learn that it's reinforcing to work toward relaxation.  I click and treat a slight drop of his head, then another.  I proceed to groom and watch his head.  Soon he relaxes, the grooming reinforcement kicks in, adds to the clicks and treats and he is calmly standing to enjoy his massage.  

Now it is winter.  It is cold and muscles are tight.  It's hard to stand still and he wants to move to stay warm.  There's a reason massage therapists have warm rooms and heated blankets on their tables.  It helps people to relax so they can enjoy the massage.  I do not have a heated barn aisle.  In the winter I use clicks and treats more to help him relax while I groom him.

But what about arena work?  What about ridden work? After people settle in their minds that yes, we do click and treat from the saddle, they fear they will never be able to just ride without constantly stopping to click and treat every step of the way.  When can we stop clicking and treating so often?  This is the question I asked Percy last week.  

I know I can reinforce one behavior by asking for another behavior that he enjoys doing- that's why I was able to reinforce walking with me to the pasture by releasing him to go play and graze.  One way I have ensured that I have lots of behaviors he enjoys doing, is by training them with positive reinforcement.  He likes to respond to my cues- he has had fun learning them and has gotten lots of reinforcement for performing them. This would not be so if he'd been forced, with pressure or equipment or fear.  That's why clicker trained horses offer behaviors- they like to do them, plain and simple.  

When I completed my training to earn my KPA CTP (Karen Pryor Academy Certified Training Partner), one of the things we had to do for graduation was demonstrate this by having our dogs perform 10 behaviors in a row, with each one acting as a reinforcer for the previous one- one click and treat at the end of all ten.  The key to doing this is to cue the next behavior at the moment you would have clicked the previous one.  What this meant was these 10 behaviors were completed in about 30 seconds!  I can't remember the exact behaviors I used for Eloise but as an example, if I cued "sit", then as soon as her fuzzy butt hit the floor, I'd cue "pop" for her to leap off the ground and as soon as her feet left the ground for that leap, I'd cue "down" so she'd land and lie down immediately and as soon as her fuzzy tummy hit the ground, I'd cue something else.  You get the picture. It was fast! But I got ten behaviors, all for one click and treat.

When we received this assignment and understood it, I couldn't wait to bring it home to Percy.  And I was thrilled to be able to come up with ten behaviors I could use, and even more thrilled when I tried it and he was successful.  Those behaviors included Alex's Foundation Lessons such as head down, following a target, stand on a mat, and Grownups, in addition to a "wait" and recall I had taught him.  

But still- what about riding?  How am I going to maintain a gait around and around the arena?  One step at a time.  I had built a quiet walking next to me behavior one step at a time. Then we'd go a few more steps before a click and treat. In working with him in the arena, I have defined the "walk on" behavior by using the criteria of ten steps.  I cued a verbal "walk on", and after ten steps, I clicked and treated.  But then I linked those tens together, so that instead of clicking after ten steps, I verbally cued "walk on" so he would walk another ten steps.  Then I clicked and treated.  He was fine with that. Remember, this is a busy boy who is not just going to walk around with me ad nauseum because I say so.  He's perfectly capable of creating his own entertainment (grabbing a bite of grass, touching the electric rope with his nose just to show me he knows the part around the arena is not on, stopping to dig in the sand and roll…these are all things he likes to do and will do if I'm not clear about what it is which will earn reinforcement).  Creating his own entertainment is highly reinforcing. I don't stop behaviors or punish behaviors.  I avoid unwanted behaviors,  by being clear with my cues about precisely what they mean.  And reinforcing them when executed correctly.  

It was quite clear he understood the ten steps.  And quite clear he was comfortable with my cueing another ten.  So I added a third ten.  Not a problem, not a bobble.  No grass grabbing, no fence poking, just quiet steps.  I added other behaviors he likes.  We'd go ten steps and then do "Grownups", standing quietly at my side when I fold my hands at my waist.  We'd do that and then do another ten steps.  We'd go ten steps and then another ten and then step over a rail on the ground.  I mixed things up but stuck to clear and happily performed behaviors.  I began to build a 30 step unit of steps by leaving out one of the cues, and then another.  He did not get frustrated or impatient or bored.  He stayed mentally connected with me so that when I cued something different, he was right there to respond to a verbal or hand or body cue.  If he had not responded promptly, I would have known that behavior did not meet criteria and I would have needed to go back and do a better job of training it. 

At the end of our session that day, I had chained 6 behaviors.  Because one behavior could now be defined as 30 steps or a duration of standing quietly, it really could be several minutes between clicks.  I plan to keep building on this, working on the individual behaviors, keeping them clean, chaining more together and defining some by location rather than just duration, so we can go for longer and longer periods of time without a click and treat- but there will still be lots of reinforcement.  
Percy creating his own entertainment by stealing the fencing mallet.


Saturday, October 11, 2014

Treat Values for Horses and Dogs

A recent conversation on Facebook sparked some conversation regarding using varied reinforcers with horses and I mentioned that I had my own theories.  Theories is probably the wrong word and I should have said "thoughts".  A couple people asked me to expand so here it is.

I believe Dr Jesús Rosales-Ruiz  did research showing that "jackpots" (for any species?) interrupted the flow of learning.  I'm not sure how the research was done, nor what was defined as a jackpot. A fellow trainer whom I highly respect, Amanda Martin, says she took the question to her horses and they told her the same.  So there are two camps that would seem to indicate that.

When I train dogs and help clients with their dogs, I do carefully use different values of reinforcers.  Following Susan Garrett's model, I make sure we have a careful ranking of reinforcers- at least ten. This list will be different for each animal, including those dogs who prefer play (or work) to food. When I ask for responses to cues in a distracting environment, I use higher value reinforcers: more difficult for the dog to do, therefore it pays better. I stock my treat pouch with higher quality treats so that every effort while working on that skill is rewarded with a high value reinforcer.  To me, that conditions that behavior more strongly with positive associations than if I used a lower value reinforcer.  Later, when the behavior is better known or easier for any reason, I start using lower value reinforcers and the higher value ones get put away for a time when they are needed.
I used to use three different levels of treat with my horses: hay stretcher pellets, peppermint flavored horse treats and wrapped people peppermints.  They were enjoyed by my horses in that order.  I never noticed a change in the training flow but I never actually documented time or any other way of assessing whether there might be an interruption.  I could easily see that the time it took to unwrap a peppermint slowed things down, and they probably did take longer to eat since the horses savored them.  What I don't know is if that slowed or in any other way negatively affected the training or whether the speed was made up for by the increased value of the peppermint once the horse got the treat.

I think many years ago I kept all three treats in my pocket and would give the horse a specific treat dependent on the quality or effort they put in each offered behavior. Now I think that is the purpose of a shaping plan; once you get the behavior, you carefully shape such that you only click the efforts which are "average or better" (another technique learned from Susan Garrett)- the treat can remain the same.  It's the clicks which pinpoint the quality. And it's my job to set the animal up so that he knows what "better" is.  I don't want a frustrated learner who does not get clicked- I need to keep the ratio of clicked efforts to unclicked efforts very high.

When the research came out saying that jackpots interrupted the flow of learning, I faded the high quality treats myself for horses and almost exclusively use hay stretcher pellets now.  
However, I do wonder, looking back, if my higher value treats with the horses were helpful.  One thing I used the wrapped peppermints for was training Percy's recall and that is still an amazingly strong behavior for him.  I also used to use it to end a session, and that did not work terribly well since it made him want more.  Now I use a handful of hay stretcher pellets on the floor as an end of session indicator.  Interestingly, emergency recalls are one place I still use high value treats with my own dogs.  As I explain it to clients, I want that dog to hear my emergency recall cue and come FLYING to me IMMEDIATELY.  I have several conversational recalls I use which mean anything from "hey, when you're done sniffing there, I really would like to continue walking" to "Hurry up, it's bloody cold out and I want to open the house door and go in".  My emergency recall is for times such as the UPS truck is coming in the driveway at about 70 miles per hour and I want you all right here by my feet right now.  Or they've disappeared into the woods after a sniff and I'm starting to worry so please return right now. For those times, my cue is a whistle and the dogs (those which can still hear) come racing back.  For that, they get the best I have to offer- I usually try to keep string cheese in my pocket but if I've got steak fat trimmings or anything better, that's what they get.  
I really do want that same reaction from my horses.  One thing I love about clicker trained horses is the ability to recall them.  I rarely use it to get them to come in for a training session.  For one thing I only want one horse at a time and it seems more polite to go get that one than to have all six come running and then only work with one, even if I did reinforce each for coming.  What they want is a training session.  But in a pinch- a gate left open by mistake for instance, or foul weather when I really do want all of them to come in, I love to be able to recall and get an immediate response.  So I think that wrapped peppermints are a legitimate reinforcer in that situation.  I'm not in a training session where I need to keep the flow going.  It's a one time "thank you!" which should reinforce that recall for the next time they hear it. 
Grass is a very high value reinforcer for horses
The other thing we have to realize is that if we get beyond simple training to secondary reinforcers, Premack and more advanced skills, we are using different value reinforcers with our horses.  Any chains we build utilizing cued behaviors as reinforcers, using grazing as a reinforcer, etc all are examples of varying the strength, variety and amount of reinforcer. We have to decide on the appropriate reinforcer for the behavior if we want to use these tools well.  
The last thing I want to mention is something fascinating which has happened with Percy.  Percy isn't what I'd call a terribly food motivated individual.  When he goes into his stall at night, he is more interested in what people and other horses are doing than his dinner.  He may take a bite of hay and walk to the aisle window to look around while he chews it.  He may stick his nose in the grain and shuffle around a bit.  He always cleans up his hay eventually but he doesn't always eat all his grain.  But he LOVES hay stretcher pellets.  Ab-so-lutely goes bonkers for them.  I have come to believe that the training process has given value to the hay stretcher pellets, as opposed to the opposite.  If he hears me scooping handfuls of hay stretcher pellets into my treat pouch, he just about climbs in the barn window.  So while I have always said I "just use plain old hay stretcher pellets" for training, in fact they could be a very high value treat for him, regardless of my opinion.  
One of MY favorite reinforcers.





Friday, April 4, 2014

Citizen Science- Why Do Clicker Trained Horses Drop?


The guest speaker at Clicker Expo was biopsychologist Susan Schneider, author of The Science of Consequences. In her closing talk, she stressed the opportunity we all have in this day and age to participate in Citizen Science.  Citizen Science is defined by Wikipedia as "scientific research conducted, in whole or in part, by amateur or nonprofessional scientists, often by crowd sourcing and crowd funding."
An example Dr. Schneider used is the Christmas Bird Count each year, where volunteers from around the country help do a bird census.  An experienced naturalist, she encouraged participants to observe the world around them and then share their findings with others via blog posts or letters to researchers or universities or any way one can think of to communicate and collaborate.


It occurred to me that an issue which would benefit from this sort of collaboration is that of dropping penises.  Many people find that having a horse going around with his penis dangling is a bit embarrassing.  And yet many male horses being clicker trained do this.  There are many guesses as to the reasons for this occurring, what to do about it and what it implies.

The important word in the previous sentence is "guesses".  They certainly aren't theories, which require a tested, well-substantiated and unifying explanation.  It would even be a stretch to call them hypotheses, since those are educated guesses and these guesses run the gamut on how educated they are.  

Some people warn that a dropping penis indicates Dominance.  The D word  has caused more trouble in animal training than any other I can think of.  I cannot follow the twisted logic that uses dominance to describe each problem every dog, horse or other animal exhibits.  I certainly cannot figure out any reasoned argument where it would apply to this situation.   

Other people think that there is a sexual component to it.  Perhaps the horse is too excited, over threshold, over stimulated by food.  Again, how does this line of reasoning proceed?  Where is the connection between food, excitement and the horse dropping his penis?  If there is a sexual component to it, why do so many the geldings do it?  I agree with those that say some real brain studies would need to be done to make this connection.  

I have also heard that somehow lateral steps initiate this and that race stables use stepping laterally to encourage a horse to urinate for a drug sample.  Others say their clicker trained  horses can do lateral work for a long time while keeping everything tucked away but this same horse will drop while being clicked for going straight.  And where is the connection between stepping laterally and urinating (and they don't always drop to urinate)?  

The time which I am most familiar with geldings dropping is when they are sedated.  They are relaxed, their heads drop, their eyes droop and their penises drop.  I would guess that this is due to loss of muscle tension.  Yet while clicker training can really help a horse relax, it also frequently perks them up and engages their minds.

Here is an interesting series of photos I just grabbed from a 55 second video I took of Alexandra Kurland working with my young horse Percy.  We had traveled to Alex's Clicker Center a year ago, fall.  Percy was very vigilant in his new surroundings- (I wrote several posts about this trip which you can read here if you like).  He looked, he paced, he worked himself into a sweat trotting back and forth, he didn't want to eat, etc.  At one point, Alex did some body work on him.  There was no special modality, just Alex doing what she felt Percy needed- and wow did it work.  As you can see in these photos, he completely relaxed- to the point of head hanging, yawning (photo 2) and finally dropping (photo 3).  There were no drugs involved, there was clicking and treating, he was free to express his opinion of the process, but he was soooo sleepy by the time she was done.  There were no lateral steps, no overstimulation from food and uh, no dominance.  
  

Percy being lulled into nap time by Alexandra Kurland.

I hope it's obvious I have no clue as to the cause of this issue.  But speaking with Alex about it, she brought up a very pertinent point in my opinion.  Lots of geldings who drop get punished for it.  Whether it's on the cross ties, working in hand or anywhere else, I've seen some pretty harsh punishment doled out.  So perhaps it isn't that clicker trained horses drop, but rather that all horses do, and we clicker trainers just don't punish, so the behavior persists, for whatever the antecedent and consequence naturally.


Bringing Citizen Science to this issue would include bringing the subject out of the closet.  We need to replace the embarrassment with curiosity, data collection, and sharing of our experiences and results.   I invite everyone to jump on this project.  I am going to create a table I can fill in quickly and easily when I observe this.  Horse, conditions (on cross ties, work in hand, being groomed?), any antecedents or consequences I can observe (what happened right before the dropping which might have triggered it?  What happened afterward that is reinforcing this?  Did the horse remain dropped or did something trigger him to pull it up again?).  And certainly note any erect component.  This is not as common but certainly does happen.

If anyone knows a graduate student looking for a thesis project, let's send them all our data and encourage them to research this!




Tuesday, April 1, 2014

Free Food and No Food- occasional training without the clicker

If you look closely, you can see that the fence is at fetlock height…an indication of the depth of the snow.
This winter has been long and cold in many places and this is one of them.  Now that March is officially over, they found that it was the second coldest Vermont March in history- the coldest one being in 1900.  All I know is that we still have feet of snow on the ground and it seems like a long time since I've done much training.  When the days don't get above zero, my hands don't last long out of mittens so clickers and treats are hard to navigate. 

Coincidentally or possibly not, I found myself working on two little training projects that didn't require the clicker, and one didn't even require treats. One was dealing with horses crowding at the gate to come in and the other was Percy's general attitude about people in his space.

While I love having multiple horses out together to be social here (at our previous farm they were more separated due to space limitations), it has meant that everybody wants to come in at once.  Having very friendly youngsters and not wanting to use pressure to back others off, I considered working on teaching them to "station" in different places (standing at a specific target away from me) but a) that meant spending time preparing that training and it was too cold for that and b) I have trouble with stations when anything put on the ground just gets covered with snow and electric fence doesn't really have rugged enough posts to hang targets on.  

Instead, I did the following.  Entering the paddock, I put a halter on the horse or pony closest to the gate, and then asked the next in line to back up a step (a known behavior for all).  I gave a tongue click and dropped a handful of hay stretcher pellets on the ground to keep him busy and back while I scooted the first horse through the gate and shut it behind us.  Then I would repeat with the next horse, and so on.  This quickly developed into a routine which the horses anticipated.  While I was putting the halter on the closest one, the next one would voluntarily back up and wait for me to come and drop treats at his feet (nothing like fresh snow, frozen hard every day to have a nice clean place to drop treats on the ground).  Since that is now being offered, I should probably put it on cue somehow…maybe a "wait" cue which Percy knows but the others don't.  In any case, it has made bringing horses in quite pleasant and calm.

The other issue was that Percy was getting quite cranky with people entering his stall.  I think partly it was the cold- everybody was cranky.  Another part was that he didn't like people moving quickly. Lastly- this is the first winter he's been locked in a stall.  Previously he could get in a stall but was not locked in, so he may have been feeling somewhat trapped and defensive.  And of course, he likes to be working all the time.  He got angry when someone just went in his stall to fill a water bucket and didn't ask him to do something that he could earn treats for.

I decided to initiate a new little ritual which did not involve any clicks or food.  I've done bits in the past of explaining to him that there won't always be treats when I show up, but this was a new situation and I wanted to be methodical about it.  If I approached his stall and his ears went back, I'd stop until the ears came up again.  I know him well enough to know that he didn't want me to leave- he just wanted to know what was going on.  But I also didn't want ear pinning to become the norm.  I needed a consistent approach that he could rely on.   Once his ears were relaxed again (usually pricked up to see me), I opened his stall door, which he knows as a cue to back up so he did.  Rather than click and treat that, I just reached out to rub his face gently.  Sometimes that irritated him so I'd freeze again until the ears relaxed.  At first I'd just barely graze his face and then proceed to fill his water bucket or pick his stall.  Any time he got cranky, I would stop all movement again until he relaxed.  

This has evolved into a new nighttime routine.  I approach his stall, he backs to let me in and then when I approach, he buries his head in my chest while I rub his face, play with his ears and stroke the sides of his neck.  He loves to lick the front of my coat while I do this…that could get soggy when I'm only wearing one layer of clothing this summer. 

While I did not use formal "clicker training" for these little projects, my knowledge of the rules of Operant Conditioning, as well as observation skills, did allow me to be successful.  

  • Animals will repeat what is reinforced. 
  • Use reinforcers the animal chooses
  • Break the training down into manageable pieces so the animal can be successful
  • Observe the emotions carefully
For the gate manners, the horses already knew the backing behavior and it was on cue and reliable.  All I had to do was ask for it.  They each love hay stretcher pellets so that was an effective reinforcer- a full handful was a jackpot.  Jackpots are currently believed by many to be an interruption in the training, but I was not in a training session, simply reinforcing heavily.  This, in technical speak, is a DRI...a Differential Reinforcer of Incompatible behavior.  If the horse was backing away from me, he could not also be crowding at the gate.   Thus, I could reinforce an incompatible behavior, rather than trying to stop the unwanted behavior of pushiness. 

The crankiness is a little harder to parse.  While there was no food reinforcer, there was the reinforcer of attention.  I had to know Percy to know that was reinforcing to him.  There was a bit of Negative Punishment going on as well as Positive Reinforcement. (my cheat sheet for those definitions are think math for Positive and Negative- one takes something away, one adds something.  Reinforcement makes something more likely to happen again, while punishment makes it less likely).  So when I froze my movements when his ears went back, I was removing something- the attention- to make it less likely he pinned them in the future- punishment.  When his ears came up again, I continued my approach, adding the attention (positive) to make it more likely he'd keep his ears up in the future (reinforcement).  

I certainly could have clicked and reinforced him for ears up on my approach…but that would have required removing my mitten to dig out treats.  Brr!

It was above 40 today and I have just returned from Clicker Expo- more stories to come.

Friday, March 14, 2014

One of those "art of training" questions: when should you stop?

The morning after
In previous blog posts, I have recommended Alexandra Kurland's Online Clicker Course, for which I am a coach.  One of the privileges of this course is the ability to participate in a yahoo group list specifically for the course attendees.  Students share their experiences and what they have learned.  Coaches write in with comments, and frequently Alex herself will write a post to everyone.  

One (of many) things I really admire about Alex is that rather than just giving a student "the answer", she invites and encourages people to think, to explore and to work things through.  There are many ways to train any specific behavior and the variables of horse, environment, history and trainer all need to factor in.  

Recently she wrote a great post and included the phrase which I have used above as my title: "one of those 'art of training' questions: when should you stop?".  New trainers and horses start with 10 treats so they have frequent resting places to consider how things are going before continuing.  Experienced trainers and horses may take a break after 10 treats, or 10 trials, or 10 minutes or half an hour or….how do you know?  That's why it is an "art".  One must use the specific situation, experience, education, and that elusive "feel" to determine whether you should quit while you're ahead or push on through a difficult spot.

I had an experience related to this yesterday.  Actually, Percy's life has been related to this topic.  In my traditional training education, "he'll get over it", was a familiar term.  "Leave him alone to settle" was a variation.  Percy taught me he can go weeks without settling if something makes him nervous.  Alex taught me how to help him settle in those situations.  

Alex and I have also shared barn building joys and nightmares in the past couple years.  I love my new barn but one little glitch is the ridge line of the roof.  The contractor designed it to ventilate, but it also allows snow in.  Every time it snows, we get a strip of snow, as deep as what we get outside, in the loft under the peak of the roof from one end to the other.  We're waiting until warm weather to see about fixing it.  In the meantime, we moved all the hay to either side of that line and simply shovel out the snow each time.  

In the last two days we got slammed with an enormous March snowstorm.  The wind blew too hard to measure it outside, but there was a swath of snow 30- 32" deep down the center of the barn loft.  My husband decided to shovel it out last night just after I had brought the horses in.  In our previous barn, there was no loft so it took Percy a while to adjust to hearing people (and the cat!) overhead, but 6 months in, he is fine with it.  My husband, however, was not just walking and moving hay overhead.  He was alternating between sweeping with a big push broom (sweep sweep bang bang as he knocked the snow off it), shoveling and scraaaaaping with the snow shovel, walking back and forth on very cold snow (SQUEAK SQUEAK) and then throwing the snow out the loft door to come filtering or crashing down outside Percy's and Rumer's stall windows.  


Mariah and Kizzy were fine- just kept munching their hay.  Rumer and Percy were wide eyed with heads high and circling in their stalls.  I could have asked my husband to stop and done it myself another time.  That would have avoided the need to work through it and I could have made that decision based on the fact that this wasn't something he'd need to deal with often. Alternatively, I could have turned him back out.  But the wind was still howling, it was dinner time, all his buddies were in and he wouldn't have been happy with that solution either. I could have left him alone to deal with it and settle or not on his own, which I have learned in the past simply does not work.  So I chose to work with it. 
The pile of snow at one end AFTER I had reshoveled much of it away from the door this morning.

I began in the aisle and just dropped 2 hay stretcher pellets in Percy's feed tub, then crossed the aisle to drop two in Rumer's tub, then back to the first again and repeated.  Each of them stopped circling to gobble up the treats.  When my pocked was empty, I went to the feed can, refilled, came back and began again.  When the second pocketful was gone, I noticed Rumer had hay in her mouth when I reached into drop the treats in her tub.  She was relaxing enough to eat her hay.  This was great because even though she's a pony who one would think should be pretty sensible, she had some traumatic experiences last fall (involving hormones and wildlife) which has left her with nervousness in her stall at times.  But she seemed to be ok now, so once the third pocketful of treats was gone, I left her alone (checking occasionally to be sure she was still ok) and changed tactics with Percy.

Since he was no longer spinning in circles, I decided it was safe to go in with him (safety first!).  Every time Alex has met him, whether at clinics or when we went to spend a few days at her Clicker Center, she has helped him to calm by engaging him in training.  Going back to my traditional education, I relate this to being told to "put him/her to work" when a horse is nervous or fractious.  I think truly good trainers do this with a similar expertise as Alex.  Too often, though, I think horses are simply forced, by strong bits, tight nosebands, martlngales and the like to Work and given no option.  They remain ticking time bombs until fatigue sets in.  

With Clicker Training, the horse has choices.  Our tools are the many lessons and choices we have given the horse in the past.  So when I entered the stall to engage Percy in training, I simply offered him the option to play, and reinforced him when he did.  I carefully chose what cues to offer.  Years ago, I would have gone immediately to Head Down as it is a calming exercise.  But reading Percy in this instance, I thought that would be too much to ask as he wasn't close enough to being calm for him to appreciate and benefit from that.  He was worried and he needed to keep his head up to feel safe for now.  Instead I simply offered my fist as a target, right in front of his nose, as I would when teaching a horse about Clicker Training for the very first time.  It was so easy, he didn't even have to think about responding- it was automatic- click/treat.  Repeated three times and we were communicating.  His head was still high, his eyes still wide, his pulse and respiration still reacting on alert and he'd startle in place when the snow came past his window.  But I had opened a crack of communication to work with.

After the three reps of the simplest of targeting I simply stroked his neck and clicked and treated.  He is very reactive to touch and I wanted to see if he could tolerate that and if it would help him relax or irritate him further.  He seemed ok with it so I alternated three reps of that with three reps of targeting, now moving my fist a little to the right or left each time.  Then I began offering the target a little lower.  He was ok with right or left but didn't want to drop his head as low as his knees to target.  That was very interesting because he was now offering Head Down all the way to the floor at times.  Sometimes after I treated and before I could cue, he would drop his head to the floor. Head Down is a known behavior.  He offers that a lot.  I rarely ask him to target my fist  down low.  So it wasn't the head height that was difficult, it was a known behavior vs an unusual behavior.  

I don't know how long it took my husband to finish in the loft.  But a swath of snow two and a half feet deep the length of a 40 foot barn takes a while, especially when it's being swept completely clean so it doesn't melt when and if it warms up.  I worked with Percy the whole time.  Each time my pocket was close to empty, I'd drop the last small handful in his tub, leave his stall and go refill.  In this way he learned not to panic when I left…I'd be back.  He'd be left with his worry for only a short time before I returned to help.  Each time I returned he was quieter.  I could have chosen to leave completely at some point, but even though he relaxed enough to turn his rump to the aisle (turning his back on the scary noise) and was working on newer and more difficult behaviors (targeting body parts, right and left verbal cues), I chose to keep going.  He needs lots and lots of practice with background distractions and this was a perfect opportunity.  I wanted him to experience that when he focused on me, the scariness went away.  I didn't want him to keep checking to see if it was coming back.

Peaceful and calm 10 degrees with the snow showing
the effects of the wind the day before.
My limiting factor was my left hand, exposed to the cold for feeding.  The temperature was about 5 degrees and even though Percy is a very neat treat taker with warm lips, that tiny bit of moisture was freezing fast.  It was becoming painful to put my mitten back on when I left for another handful.

Fortunately, just when I thought I couldn't take it any more, the noises upstairs stopped and I heard my husband coming down the stairs.  A couple more reps, a peppermint from the other pocket and I could leave Percy to eat his hay in peace without worrying that he'd get worked up again.  

Monday, December 30, 2013

A Day's Worth of Reinforcement

In the September issue of The Whole Dog Journal, Pat Miller wrote an article called "Daily Training" in which she detailed how she reinforces her dogs throughout each and every day.  At the end, she encouraged readers to do a daily tally of reinforcers and submit them.  I thought it was a wonderful exercise and went through it myself, but never submitted it. So now I'm going to do the same with the horses and share it here.  

Many people begin clicker training by having "training sessions" and indeed that is a good place to begin- get organized, count out your treats, have a plan, train and then assess how things went.  Once you and your horse discover how great this system is and you become a smooth handler, you can begin using clicker training in other situations and at this point, you become one of those people who always has treats in her pocket.  I can no longer say I always have treats in my pocket because although I stuff them frequently, I also dole them out frequently and before you know it, my pockets are empty again.  So I have to go refill.  

How do I go through so many treats?  Here's a typical day, keeping in mind that it is mid-winter and "typical" could look very different in mid-summer:

The very first thing I do each morning is turn horses out.  Messy stalls are a pet peeve of mine and I like to get them out asap!  I approach the barn at the exterior dutch doors, calling a greeting so they know I am coming.  I open Percy's door first.  Because he has had so much practice at backing when I open a door, I am now actually working on getting him to come out, rather than stay in.  I have to hold the door so it doesn't blow in the wind so It's challenging because I can't target him all the way out or I'd have to let go of the door.  I get him started with a fist target and then use a verbal "walk on" to keep him going.  When he walks out far enough for me to shut the door behind him, that is his first click and treat for the morning.  I then shut his door and let Mariah out.  She comes out very politely and continues out to find the hay I have scattered in the pasture- her own reinforcement.  Percy follows her.  

The ponies on the other side of the aisle need to be led across the aisle and through Mariah's stall to be turned out (well, that's the way I do it).  They are still learning about backing when I approach a stall door (having lived outside previously).  I rest a hand on the door and wait for a step back, click, open the door and treat.  Kizzy is great about going with me if I just use a hand under her jaw.  But she can be a little quick.  So I take a couple steps, click and treat for staying at my side, before she gets ahead of me, take a couple more steps, click treat, etc.  

Rumer and Ande love to investigate the barn and aren't as focused on getting out to the hay piles.  A hand under the jaw doesn't always work well with them.  Sometimes I use a rope around the neck and reinforce every couple steps as with Kizzy.  Sometimes I put a halter on in which case, they get a click and treat for pushing their noses down into the halter when I put it on.  Ande is very good at this, Rumer prefers if I stand in front of her so we're working on haltering from the side.  I do a lot of clicking for ear work this time of year.  There is so much FUZZ and HAIR on ponies in midwinter it's sometimes hard to even find the ears in order to get halters over them.  And sometimes they are extra sensitive due to the cold.  So ear handing gets reinforced a lot.   When we get outside, Ande likes to gallop off to the others and to the hay piles once released.  So he gets reinforced for turning around and waiting quietly while I remove the halter or rope.  I click the turn around and feed while I remove the halter over his ears.  In that way, he both stands until it comes off and he associates the halter removal with quiet standing, rather than twisting his head to get away.  

Rumer, on the other hand, would much rather stay and play with me when I let her go, so I don't reinforce her…it's all about balance.  I want her to move right off so she doesn't get chased by one of the other horses.  Nobody chases Ande.  Stowaway is his slow and steady self, who likes to be tugged everywhere he goes.  So he gets clicked and treated for stepping off when I do, rather than lagging behind.  

We are covered in snow and ice these days, so there is no riding or longeing possible.  I come up with different projects for the winter.  The current project for Percy is his husbandry skills.  I regularly practice for things like worming, injections and blood draws (see these two blog posts for more on that).  

At noontime, the horses all get more hay.  With anything resembling a plant now buried under snow and ice, they get pretty excited when they see hay coming.  The get lots of clicks and treats for walking politely beside me until I drop it, rather than ripping it out of my arms.  

Two days ago, my husband and I were working on the fence (which was supposed to be a permanent fence by winter but too many other things were ahead of that on the list and then the ground froze).   Since Percy and Rumer were both entertaining themselves by chewing on the electric rope, they eventually took it apart and let themselves out three times in 2 days.    I had locked everyone out in the big field with hay and then hung some strands of surveyor's tape on the gate when I closed it, to make it more visible since it had been left open in recent weeks.  When they finished their hay, they came over to the gate and Percy promptly grabbed the surveyor's tape and ripped it off.  It landed in two pieces under his nose.  Envisioning a colic surgeon removing surveyor's tape from his gut, I dashed over to pick it up.  Percy thought I was coming to play so he stepped forward to greet me.  Because he was in a corner, I couldn't ask him to back up and I couldn't figure out how to get him to move.  Being human, I said, "You need to get out of the way so I can pick that up" and pointed to the tape.  He understood the point.  And he reach down and picked up a piece of the tape and handed it to me.  Good thing I had treats in my pocket at that moment, though my laughter would have sufficed.  So I pointed to the other piece and he picked that up as well.  This time, however, he lipped at it in transit and the whole thing went in his mouth.  More visions of colic surgery…he didn't seem to know how to drop it when it was all inside.  I quickly gave him my "open your mouth" cue and like a good patient at the dentist, he opened wide.  I was able to reach in and retrieve the soggy tape from the back of his tongue.  Disaster averted.

At afternoon chore time, it can be tricky getting Mariah in because Ande wants to be first in.  To keep this from getting too long, I'll avoid telling you why that's not possible and just say I ask him to back away from the door and stand to the side while she goes in- clicks and treats for Ande, and dinner waiting in Mariah's stall to reward her for going by him when she'd rather not.  Then I have to do the same for Percy.  He isn't afraid of Ande, but nonetheless, I have to keep Ande out while letting Percy in.  When it is Ande's turn, he backs away from the gate and waits while I enter for another click and treat. Sometimes I put Stowaway's halter on first and sometimes Ande's.  Clicks and treats for putting noses in halters, clicks and treats for waiting patiently while the other gets a halter on.  And again for standing quietly when halters are removed inside.  

When I lead Kizzy to her night paddock, we are working on getting that thick little Welsh pony neck to bend into a hip give halt.  She can outwalk and I'd almost bet out pull any horse on the farm.  When she's headed for dinner is a great time to work on bending to a quiet and graceful halt for treats here and now rather than booking it to the supper in the feed tub.

Sometimes at late night chores, I'll work with someone in their stall- if blankets need to go on or come off, that is another opportunity to reward standing quietly.  My biggest problem with blankets is that all the horses are so sensitive to being asked to back that I often get backing when I'm fiddling with chest buckles.  That is an opportunity to click before they move so they understand that standing is what is wanted.  

So there is a day full of reinforcement.  Is any of this necessary?  Depends on who you ask.  Anyone could come and do chores for me without using treats to do anything.  I could as well, and sometimes I do.  But maintaining behaviors with daily reinforcement keeps them clean and prompt.  Also, there's always room for improvement in daily behaviors.  







Sunday, December 15, 2013

Four Horse Recall


A quick success story this morning:

The scene- I had just let four of the horses (two ponies) out into a foot of fresh snow and fifteen degrees.  Normally I put hay out the night before so it's waiting for them in the morning, unless, as in this situation, the weather made that impractical.  I hate leaving horses in their stalls while I put hay out because the stalls get messier.  So there they were, romping in the cold fresh snow, galloping about looking for their hay piles.  My husband offered to put a bale on the sled and drag it out to the far side of the field where we like to put it to encourage the horses to move and spread the manure more evenly in the field.   I gratefully accepted and I went out the other door to take hay to the other ponies in the shed.  

When I was done, I saw that we had a problem.

Instead of going around the field and tossing or carrying hay in, Ed had struck out across the middle of the field.  Four horses were grabbing at the hay bale as they chased it and him across the field, yanking the bale off the sled every step of the way.  My husband can swear a blue streak when he's in a good mood- he was not presently in a good mood.  They were all about 100 yards from me but I took a chance.  I ducked into the field, stepped into the run- in shed and yelled Percy's and Ande's names.  I couldn't believe they could hear me over the wind and Ed's expletives, but they did.  When they turned to look, I stuck my arm out to the side- heavily gloved hand in a fist.  

And they came.  At a gallop, kicking snow up all around them, they came right to me, leaving a full bale of hay and the fun of yanking it out of its strings, when they hadn't eaten since 10: PM.  I had one handful of hay stretcher pellets in my pockets. They each got two on arrival.  I then rationed them out, one at a time, for pulling their noses away from me.  Mariah and Stowaway had returned at a more sedate pace, and then I had four horses giving me polite "Grownups" positions, in exchange for one hay stretcher pellet at a time, taking turns with each other.  Fortunately the hay pellets lasted until the hay was distributed and even though I didn't have leftovers to put on the ground when I left, they seemed happy to wheel and race each other back out across the field to their waiting hay piles.  

Success!