Tiel Town
Drs. Foster and Smith Inc.

TielTown

Clicker Training - Positive Reinforcement Training

There's a lot of debate amongst trainers on whether to use a clicker or a voice bridge. At last year's ClickerExpo, I read a summary of a study on the effect on speed of learning that different bridging stimuli have.

The verdict? Naive shelter dogs trained to cross a room and touch a target using a clicker learned the task 40% faster than naive shelter dogs trained to do the same task by the same trainer using a voice bridge ("Yes").

Now, you can read the whole study for yourself. It's Lindsay Wood's masters thesis, and it's posted on here.

Clicker Bridging Stimulus Efficacy

By Lindsay Wood, MA, CTC

Abstract

Acquisition of a multiple component task, such as approaching and touching a target apparatus on cue, plays an important role in animal training and husbandry. Experimental training of two groups of 10 naïve dogs (Canis familiaris) to perform the target task differed only by the assigned bridging stimulus: a clicker or spoken word "good." Although both types of bridging stimuli are used in the training field to indicate the precise instance of correct behavior, this study represents the first systematic comparison of the efficacy of these two types of bridging stimuli. There was a decrease of over 1/3 in training time and number of required reinforcements for the clicker as compared to the verbal condition group. The clicker trained dogs achieved behavior acquisition in significantly (p < .05) fewer minutes and required significantly fewer primary reinforcements than verbal condition dogs. The difference in effectiveness of the two bridging stimuli was most apparent at the onset of each new task component. It appears that use of the clicker, by providing a more precise marker than a verbal bridging stimulus, is responsible for superior acquisition of complex behaviors such as that studied here. The facilitation of learning provided by the clicker bridging stimulus has important implications for animal training, especially when professionals are confronted with time constraints. The potential of the clicker stimulus to improve animal learning throughout the entire process of a behavior may not only increase the rate of behavior acquisition, but also reduce animal frustration and further enhance the relationship between trainer and animal.

My dogs both know what the clicker is. This poses a couple problems. I can't work with them at the same time, and they sound like they are being killed if one is being worked with and not the other.

So I bought a clicker+. It has 4 different sounds you can use. I want to use the Ping with Max, and then continue to use the iClick with Bindi.

So I need to get Max to realize that the traditional click, isn't his sound. So I "charged up" the Ping with him and have been using that. I will also click/treat Bindi and ignore Max, then I'll ping/treat Max and ignore Bindi.

The only other idea I've got is to put Bindi outside and let the girls go town with clickers, but that would probably be pretty frustrating for Max.

The easiest solution would be to train two completely different sounds, giving each dog a new one. But then I run into the problem when working together that I'd forget which click the clicker+ was on, and click the wrong dog at the wrong time. It does switch pretty easily between two sounds though.

Is this even possible? or is just going to cause so much confusion for Max that I should forget it and buy a second Clicker+? That's the only way two completely new sounds would work for me. Timing is still something I'm working on, and I don't want to make it worse by fumbling with the side switch to get the right tone. With two I could have one in each hand.

Hi all! So I've decided to start clicker training with my caique. I've already done a bunch of reading on positive reinforcement and have read Melinda Johnson's book on clicker training. I've also did the food test. He loves bits of popcorn and walnuts, but generally speaking, he'll go after any food you put in front of him.

So yeah, I'm just about ready to start clicker training with him, but where exactly should I start? Should I get him used to the clicker sound? Or just start working with him with the clicker. Do you recommend I do something easy with him like step ups, something he knows but sometimes just refuses to do or should start with target training first. When it comes to using a target, what do you recommend using? A small stick, a pen? Does it really matter?

Sorry for all the questions. I'm just a little nervous and don't want to mess anything up.

Thanks!

I've had my two parrots for about two and a half years now. For most of that time I've been working on "potty training" them - that is, 1) poop on cue, and 2) poop off cue in appropriate places, like the top of their cage.

They got part 1 down pretty darned well, but we've been working on part 2 for a loooong time. We've achieved a pretty happy balance where if we humans remember to cue the birds to fly home every 15 minutes or so, they'll poop on top of their cage. If we don't remember, they'll poop wherever they happen to be, including on us.

Last night, Daedalus, my smart little girl-bird, flew back to her cage without cueing and pooped. Not just once, but at least four times over the course of the evening! (I was there for two of them.) Needless to say, she got a huge treat for that.

I am SO PROUD!!

Now if only I can get her dopey man-friend to do the same... ;)

Tomorrow my parrots and I are giving a little talk about (and demo of) clicker training to an elementary school class. I'm trying to think of how best to explain positive reinforcement to kids, as well as why it's more effective than punishment.

Does anyone have any good examples I could use? What are good real-life examples of positive reinforcement that will resonate with 2nd and 3rd graders? (Examples of punishment are welcome too.)

Any tips and suggestions are very welcome! :)

ClickerTraining.com is having a sale on all of its interactive dog and cat toys. Check out their Boredom Buster toy page.


I'm working from home today, and my birds are out and flying around the house amusing themselves. A few minutes ago I heard a clicking noise, and I looked over to find Icarus, my white-bellied caique parrot, sitting on the bookshelf triumphantly beaking the end of the target stick, which I'd left sitting on that shelf weeks before!

I cracked up and gave him a treat. What an industrious little clicker-trainee! :)

(x-posted to my journal)

Just in case you guys haven't seen this yet:



This is interesting, since it gives us more information about animals' emotional states. Karen Pryor actually talked about this experiment at ClickerExpo last year.

Check out this article about clicker training cats from the San Francisco Gate!

Clicker training does more than just teach cats what we do and don't want them to do, says Redwood City cat behaviorist Marilyn Krieger; it enriches their lives and our relationship with them. "I have one client whose husband is not in love with cats," she told me. "I have him clicker training, and already he's appreciating his cat more. There's just something that happens, where they become more likely to interact with you. There's more communication, and the cats seem to understand so much more what you want."

Clickertraining.com just posted an article about training your dog for husbandry behaviors (e.g. nail clipping, vet exam, etc.

I am curious about how many of you have trained your animals for husbandry/veterinary behaviors? If so, which ones? Which behaviors do you think are most important?

(I'll post my response in a comment.)

Hello all!
I finally got around to getting back to work on Bonsai's clicker training! I took a break for a while because he just wasn't responding too well to getting a food reward... but now that I've switched him over to a pelleted diet, I can use his seed as c/t treats and it works just wonderfully! Today, while we were training, Bonsai was so charming. I started by doing several reps of "step up" because he is already quite good at this one, and then moved on to "come here." When I moved to "come here" he would say "step up" to me as he was running towards my hand, it was too adorable! But of course, when I break out the camera to catch it to share with you all, he wouldn't do it anymore!

Anyway, I still recorded and posted it on youtube. Bonsai has an interesting view of physics that I think you all might find interesting. ;)
Such a clever p'let, he is. XD


x-posted, sorry if you see this more than once!

I hope that everyone enjoys their 4th of July and gives their puppy an extra treat in light of the festive day.
You'll probably all be eating extra food so give them an extra bite or two.
X-posted

Hi, I wanted to get some feedback on my guide dog in training Laveau. Just for general information; she is a one-year-old Lab Doberman cross.
Laveau likes to swing her butt out when she is walking or when she sits.  Sometimes it's an ever-so-slight swing but sometimes it can be very pronounced.I
don't recall ever stepping on her and I think she's been doing this since I got her.  I'm really noticing it since I've been introducing the harness to
her over the last couple of days.  I am going to start by working with her in an area where she can't swing her hips out and has to keep parallel with
my leg but is there anything else I can try that will make her more aware of her hind end and to bring it in closer to me?  
thanks

My standart poodle (hairstyle "cropped"))) Not anything special but pretty funny.


"Hello!"

"Two paws" and "up"

Do you have any good links for training tricks in parrots, or training in general?

I'm particularly interested in a good description of how to teach targeting as well as any of the cute tricks.

They need to be freely available web pages, not books or videos, for what I need them for, alas.

Cross posted to [info]clickertraining and [info]parrot_lovers.

Daedalus has mastered the Coins in the Piggy Bank trick! She got it so quickly that I felt like I was just along for the ride most of the time. This video is actually the first time she wasn't getting reinforced for every coin, and she isn't phased at all.



(x-posted to [info]clickertraining, [info]parrots101, and my journal)

I bought the game You Don't Say while I was at ClickerExpo, and finally played it recently. I am pleased to report that it is AWESOME!

I found it much easier than the regular "clicker game" (aka free-shaping a human using a clicker) because it's limited in scope: because all of the behaviors are related to the set of small objects laid out on the table, the person being trained has a starting place to work from. S/he can touch all of the objects first to figure out what the trainer wants her/him to work with.

It's a great opportunity to be the trainee, as well, and feel what it's like to "get it," or to get frustrated, or to try to outthink the trainer and jump ahead.

I'll bet this game would also be a great way to give people an introduction to the principles of behavior. I could see it being used in a behaviorism class, or for teaching kids how to train the family dog and to think more broadly about the effects their own behavior has on others.

I can't believe I didn't post this the first time I saw it! This video really makes my day:


Over the last couple of days I've been teaching Daedalus, my black-headed caique parrot, to put wooden "coins" into a small piggy bank. To my astonishment, I'd barely put the bank and coins on the table for the first time when she picked up a coin and took it right over to the pig. She immediately knew what I wanted, despite never having seen anything like the bank or coins before. I swear, sometimes those little parrots are scary-smart!

Daedalus' accuracy at getting the coins in the slot, however, is not great yet. It's hard work maneuvering the coin into the bank, so she often tries to maneuver the coin into the slot but doesn't quite make it, and the coin ends up sliding off of the bank.

Here's my question: I know Daedalus knows what I want the ultimate behavior to be (putting the coin through the slot), but she doesn't manage to do it very often. If I only reinforce completely correct reps, the rate of reinforcement (i.e. number of treats per minute) is too low and she gets bored and flies off. But I don't want to reinforce all of the failed attempts and confuse her or end up training her NOT to put it in the slot. So I've started using two different reinforcers: lower-value reinforcers (sesame seeds) for the close-but-not-quite attempts (which I'm shaping), and a jackpot of high-value reinforcers (dried bananas) when she actually gets it in the slot.

Does that sound reasonable, or is there anything I should be concerned about if I use this approach? I'm pretty sure I've read about dog trainers doing this, but I'm not 100% sure. Any input is appreciated! :)

(x-posted to [info]clickertraining, [info]animal_training and [info]positivelydog)

For those of you who keep fish, The Fish School sells a cute kit for teaching your fish "agility"-style tricks. Looks like it's geared towards smaller fish, but it's a neat idea nonetheless.

This is one of the coolest applications of behaviorism that I've ever seen.

I've heard a lot of people talk about using praise to train their animals. Instead of giving a treat, many people say "Good dog," "Good bird," etc.

Whether they know it or not, these people are using secondary reinforcers. While secondary reinforcers can be very effective, they should also be used cautiously.

Here are my notes from Ken Ramirez's session on Using Non-Food Reinforcers at ClickerExpo KY 2008.

What is a Secondary Reinforcer?
First, let me define our working vocabulary.

A reinforcer is anything that, when paired with a behavior, causes an increase in the frequency of that behavior. A reinforcer causes the reinforced behavior to increase.

This definition is important because many people make assumptions about what are and are not reinforcers for their particular animals. (See the "Good bird" example above.) Don't make assumptions about what your animal likes!

Before you consistently use any reinforcer, it's important to test it to make sure it's actually reinforcing to the animal. Do this by using it to reinforce a behavior and see if the behavior increases. If so, you've got a valid reinforcer on your hands.

A primary reinforcer is something that satisfies an organism's biological need. Common ones are food, water, air, sex, and shelter. There are others, of course, which vary from species to species, but this group is pretty universal. Since most of them are difficult to use to reinforce behavior, the most commonly used primary reinforcer is food.

A secondary reinforcer (also known as a conditioned reinforcer or reinforcement substitute) is something that increases behavior without fulfilling a biological need. Secondary reinforcers work because they have previously been paired with a primary reinforcer.

For instance, if you always say "Good dog!" when you give your dog a treat, eventually the dog will see "Good dog!" as a reinforcer. For those who train with clickers, a click can eventually become reinforcing on its own.

The concept of secondary reinforcement is based on Premack's Principle. (This blew my mind the first time I learned about it.) Premack's principle, in a nutshell, is that a more frequently performed behavior (or more willingly performed behavior) can be used to reinforce a less frequently performed behavior (or less willingly performed behavior). For instance: kids in school might be more eager for Art class than Math class. If they know they don't get to do Art class until they've finished their Math exercises, they'll be more willing to get through the Math. Art is being used as a reinforcer for Math! The kids might enjoy recess or snack time even more than Art, but that's irrelevant - Art, even though it's "work", can be used as a reinforcer for doing Math.


How Does One Use Secondary Reinforcers?
All secondary reinforcers should be trained like any other behavior, through repeated pairing with a primary reinforcer. Remember that secondary reinforcers are learned behaviors, just like every other behavior. The more a secondary reinforcer is used without being followed by a primary reinforcer, the weaker it will become as a reinforcer.

When using secondary reinforcers without pairing them with primary reinforcers, it's important to be careful not to over-use them. A good rule of thumb is to follow the secondary reinforcer with a primary reinforcer 80% of the time, and only use the secondary reinforcer on its own 20% of the time. This will keep the strength of the conditioned reinforcer high, keeping its effectiveness intact.

A word of caution: DO NOT use secondary reinforcers too much or too soon. Before using secondary reinforcers at all, one should have a well-established relationship with an animal and be well aware of its personality and preferences.

Just because you see someone else engaging in a certain kind of behavior with an animal or using a certain secondary reinforcer doesn't mean that you can use it too. Secondary reinforcers are very much dependent on a relationship between two organisms. Think of it this way: Your husband (or wife, or partner) might show his appreciation for something you did by giving you a kiss. It would probably make you more likely to do whatever it was again! But you'd never (hopefully!) want your boss to reinforce you that way for doing good work. From a boss, the secondary reinforcer of money is much more effective.


Here's an illustrative story that Ken Ramirez told us:
When he was first working as a marine mammal trainer, for the first couple of years he wasn't allowed to use ANY secondary reinforcers at all. When an animal performed a behavior, it got a fish. Period. During his first year at the aquarium he spent time training the animals and getting familiar with them, but wasn't very involved in the actual shows. He basically spent show time kneeling on the deck smiling and observing.

One day the head trainer cued the animal in the show, a whale with whom he'd worked closely for twelve years. As Ken watched, the trainer cued behavior after behavior, rewarding the whale with only secondary reinforcers like patting and tongue-scratching. Ken watched, amazed, as this senior trainer kept cueing behaviors for five, six, seven minutes. After ten minutes, the trainer finally cued the whale to toss him onto the deck. He turned around, gave the whale a huge hug, asked him to open his mouth and scratched his tongue vigorously. He then picked up a whole bucket of fish and poured it into the whale's mouth.

Behold, the amazing power of secondary reinforcement! Ten minutes of behaviors without a single primary reinforcer until the end!

Later that day they did the whale show again with the same animal. This time, though, a relatively novice trainer only about a year ahead of Ken was cueing the whale. To Ken's surprise, this trainer didn't reinforce the whale with fish, but instead used secondary reinforcers! Ken watched as the trainer cued behaviors this way for five minutes, then six. He realized that the trainer was going through the exact same routine the senior trainer had done that morning.

After ten minutes of behaviors, the junior trainer cued the whale to toss him on the deck. He turned around and gave the whale a big hug. He asked him to open his mouth and vigorously scratched his tongue.

And the whale snapped his jaws shut and severed both of the trainer's hands at the wrists.


This story really socked me in the gut. Even though I don't train any animals who could grievously harm me, this reminds me not to get lazy about it. Every animal with a mouth bites, and it's our job as trainers never to push them to bite.


Animals develop expectations about reinforcement, just like humans do. If one day your boss said to you, "I'm not going to pay you money this week, but instead I'll give you a big hug and a gift certificate to Chuck-E-Cheeze," you'd probably be pretty ticked off. (I know I would!) Similarly, an animal that's always been reinforced with food and suddenly gets reinforced with something else is likely going to be disgruntled. This is why it's important to introduce secondary reinforcements carefully, through conditioning.


How To Condition Reinforcement Substitutes, a la Ken Ramirez
Do each of these steps many times before moving on to the next step. The goal is for the animal to be reinforced by the secondary reinforcer, not feel cheated out of its usual treat.

First, pick useful, easily available stimuli such as petting, clapping your hands, etc. It's helpful if the animal already likes it (e.g. petting), but not essential. (To illustrate this, Ken conditioned his dog to accept clapping as a conditioned reinforcer. It worked like a charm.)

Train each "future reinforcer" as you would any other behavior: Present the new stimulus and reinforce it (no click needed - save your clicks for teaching actual behaviors). Repeat, repeat, repeat ad nauseam. Continue until it is clear that the animal recognizes the new stimulus and anticipates the reinforcement.

When you're ready to try it out, ask the animal for an easy, well-established behavior. After the behavior, click, use the conditioned reinforcer, then follow it with a primary reinforcer. Do this only a couple times per session.

Next, cue a behavior, click (after the behavior, of course), and follow it with the new secondary reinforcer. Now cue a different behavior and follow this one with a primary reinforcer. Again, DO NOT overuse this! Do it only three times per session, and always with different behaviors.

Next, cue a harder but well-established behavior, click, and give the new secondary reinforcer followed by a primary reinforcer. Repeat, repeat, repeat. Continue this process with all established behaviors.

When ready, cue the difficult behavior, click, and use only the new secondary reinforcer.

Gradually, slowly, increase your use of the new secondary reinforcer. NEVER let the ratio of secondary to primary reinforcers go above 20/80.

For all new behaviors and new situations, use primary reinforcement. When in doubt, use primary reinforcement.

Don't forget to continue to pair the secondary reinforcer with primary reinforcers frequently to keep that association strong.

Rules for Using Secondary Reinforcers
- Never use secondary reinforcers after two consecutive behaviors
- Avoid using the same secondary reinforcer twice in succession
- Always follow a given behavior with a primary reinforcer more often than a secondary reinforcer. Keep that behavior-primary R association strong!
- Continue to train the secondary reinforcer forever. Use it as a behavior MORE OFTEN than as a reinforcer.


Reinforcement Pitfalls
- Don't take your reinforcers for granted.

- Base your reinforcers on the animal's preferences, its health, its level of satiation, the time of day, the difficulty of the behavior, etc. A dog may be toy-crazy, but even its favorite toy may not be reinforcing enough for a really difficult or unpleasant behavior (e.g. nail clipping or ear cleaning).

- Maintain the strength of your reinforcement substitutes constantly.

- Reevaluate the effectiveness of your reinforcers frequently, particularly for secondary reinforcers.

- Reinforcement variety helps keep the animal's attention and motivation high, but only if the animal's expectations are being met.


Have fun, and don't get bitten by a whale!

I recently realized I'd started incidentally using a few cue words with my parrots without having consciously trained them. Once I realized it, of course, I formalized (in my head) what behaviors I wanted paired with them and worked on them a bit.

The cues are "off," "out," and "go," which all mean (basically) "Get off of what you're on an go to a sanctioned perch." I tend to use "off" all over the house when they're getting into things they shouldn't (perching on the edge of the tortoise tank, climbing on the pantry shelves, etc), "out" to get them out of the pantry (which actually involves flying from one sanctioned perch to another), and "go" to mean "fly off of me onto another okay perch."

The three cues are almost interchangeable, since they all boil down to the same basic behavior, and I really don't care whether the parrots ever learn the distinction. Since they're normal-language words I use with them every day, I want them to be able to recognize all of them as semi-formal cues, but the behavior can be basically the same.

I realized that it's actually an incredibly handy behavior to have on cue, so I worked on consciously reinforcing it for a while to solidify it. It's nice to be able to say "off" and have the bird hop off of the edge of the tortoise's box without my having to actually put down my knitting and get up out of my comfy chair. ;)

I've always been fascinated by unconscious training (both human training of animals and animal training of humans), so it's neat when I can catch myself doing it (or falling victim to it). In college I had a pet rat, Sebastian, who I trained almost entirely unconsciously/informally. He knew a handful of cues - go home, recall, potty training, etc.

I'd love to hear other people's stories of "unconscious" or "informal" training. I know you've all got them!

Natural Encounters, Inc has a truly great FAQ page that answers a lot of frequently asked parrot owner questions.