Wednesday, May 10, 2017

Shocking News About B.F. Skinner




I always smile a little when I hear clicker trainers talk about the "science" of dog training.

The transparent message is that people without science degrees of any kind are now armed with the cloak of science because they went to PetCo and got a clicker, read some semi-sensible training tips on a list-serv, and have a few cubes of cheese in hand.

Why do I smile? Simple. You see, to the extent there is a "science" to dog training (and I will let others debate the semantic edges there!), it was sparked by B.F. Skinner who used "Skinner Boxes" to teach animals to press levels, guide bombs, play tic-tac-toe, and dance in circles.

Left out of the story, however, is the fact that Skinner boxes had electric floors and could administer mild electric shocks to rats, monkeys, and other animals inside. Please notice the power cord and the electric floor grid in the "Skinner Box" diagram at top.

Of course the fact that B.F. Skinner jolted animals with electricity is hardly surprising. After all, the three core parts of operant conditioning (which were well understood by circus trainers long before B.F. Skinner named them) are rewards to encourage behavior, doing absolutely nothing to extinguish behavior, and engaging in "punishment" to discourage behavior.

So, to put a point on it, if you insist on calling yourself a "scientific" dog trainer, be sure to show me where you plug in the electric grid, or how you administer your mild aversives.

Science is not philosophy -- it is the opposite of that.

Science, like Mother Nature, is not particularly soft. In fact, it is more likely to be red in tooth and claw than warm and fuzzy. Every dog comes with teeth to instruct. Not a one carries a clicker.

As for Skinner and the Skinner Box, has no one else noticed that he never trained predators, and he never worked with animals in an open -field situation?

And why not?

Simple: predators have strong prey drives, and rewards-only training does not work very well to stop prey drive. In addition, rewards only training is subject to sudden failure in an open-field situation where stimulation and distraction can come from any and every direction.

And so Skinner tended to focus on pigeons and chickens, and the occasional rat, and he spent almost all of his energy getting them to do tricks for food, rather than getting them to stop naturally self-rewarding behavior using aversives.

One reason Skinner did so little with aversives is that birds are not easily shocked through their feet, and body feathers prevent direct skin contact with metal elsewhere. Unable to easily jolt a bird with an electric current through their toes, and using only animal subjects with very low prey drive, Skinner generalized a theory of learning that works well for training tricks in a sensory depravation chamber, but which too often fails in the real world.

The simple truth is that getting a prey animal in a closed and captive setting to do an unnatural behavior without distraction, is almost the exact opposite of getting a predator, like a dog, to stop doing a self-rewarding behavior in the home, yard, or field. Any wonder then that rewards-based training, based on Skinnerian theories, so often fails outside of the trick training arena?

14 comments:

PipedreamFarm said...

So did Skinner use shock in teh same way that gun dog traners use shock collars? Meaning clicker and shock collar training both came from Skinner.

PBurns said...

Yes. A Skinner box (the device that the Breland's used to train so many animals) was rigged for food and electric shock. You could get almost all behaviors expressed through food alone (Question: is extreme hunger a cruel aversive?), but the combination of shock and food was faster and in most cases increased the consistency especially if food was no longer a strong trigger.

The great thing about food rewards, as opposed to shock, is that before shock collars came along you could give food rewards anywhere. In addition, when the first e-collars came along, they were not very sophisticated and could nonly give pretty strong shocks and no tones or vibrations.

The main positive claim (a true one!) for food-based training is that it's harder for a really bad trainer with no experience and no desire to read or get any, to screw it up too bad. An e-collar is like a shotgun or a pistol -- a very good tool in the right hands. A clicker is like a ham sandwich - hard to hurt anyone with.

But, to come back to it, YES electric shock for training is what Skinner REALLY brought to the table. Food training has been around since Moses. See >> http://terriermandotcom.blogspot.com/2011/06/old-school-dog-training.html

Stoutheartedhounds said...

"You could get almost all behaviors expressed through food alone (Question: is extreme hunger a cruel aversive?)"

So, in reference to my previous question in the other thread, are you saying that R+ and P- CAN do everything without P+? Is there anything that R+ and P- cannot do? Is it really a simple issue of efficiency?

I am not a professional dog trainer, but I do use P+ and aversives to train my dogs. In some instances I will admit that I choose to use aversives because they work efficiently, but I do believe there are certain things that can only be taught with aversives (which I already mentioned in a previous thread).

SecondThoughtsOptional said...

Eating is existential. Animals will do just about anything to meet the existential need. However, once that need is met...

One of the not so great things about dolphins is that an obedient dolphin is one short ten pounds of fish. A dolphin + ten pounds of fish is one that's going to blow you off, at least until it's hungry again. How to keep one working in the field? With a muzzle, so it can't go off and catch its own fish. There's no question that when people interact with the dolphins, they do so nicely. However, to look at the big picture is to see a far less rosy situation.

Since we actually have to live with our dogs and can't keep them continually hungry (at least, I hope not) to ensure their cooperation, we need a bit more give and take.

Miss Margo said...

Skinner's research was truly top-notch. "Inducing Superstition in the Pigeon" is one of my favorite articles.

@SecondThoughtsOptional: It's the same with parrots as you say that it is with dolphins. To harness their full attention and motivate them, you have to delay feeding them half a day, and then give them treats for performance in the training session (of course, the birds are fed full meals afterward).

This man demonstrates (well-executed) clicker training with a hungry, socialized parrot in this video: http://www.videosource.tv/animals/kili-senegal-parrot-flight-training-recall-second-day

It's awesome; his bird can do about two dozen tricks now.

My Senegal HATES to have her nails clipped, but if I do it when she's hungry and give her a nut every other toe and immediately afterwards, she endures it and doesn't even hate me after it's over.

BTW Terrierman, I always appreciate the way you show compassion for captive parrots on your blog.

dp said...

First of all , not all breeds have the same temperament. Terriers are tougher. Second of all, if clicker traing is going to be effective, you need precise timing. Third of all, if there is greater motivation than a food reward, forget it. I am talking about my Parsons, which have great prey drive.

Jacob L'Etoile said...

If you want to see a purely positive training methodology with a multi thousand year history of success on high drive free roaming predators, many of which can seriously hurt the trainer just take a walk down to your friendly neighbor hood falconer. There are certainly constraints on what can be done, and when freedom can be allowed but the methods work, very well.

Stoutheartedhounds said...

And a lot of falconers lose birds that don't come back to them. If you want to talk about something that always gets you what you want then falconry is not the answer, lol.

PBurns said...

.

Actually, falconry is mostly "the art of hunger" and there's a lot less "positive" than some would think.

Negative reinforcement is pretty much ongoing with "relief" offered as the reason to fly (to catch food), and the reason to come to the lure (after the food has been taken away).

There's not much "positive" to fierce hunger based on denial of food and sub-par body weight.

Think about it in human terms: If we don't feed you for three days, and then ask you to climb a ladder for a peach, but then take it away as soon as you get to the top, is that "pure positive" training or the kind of cruel coercion and torture that falls neatly into the folder of "fucking with the slaves"?

A hawk or falcon that is not very, very hungry is in danger of laying up at the top of a tree, or flying off over the horizon; that's why falconers weigh birds twice a day during the hunting season, why they know the bird's weight within a fraction of a gram, and why they RUN to get a bird off a rabbit or pigeon before it eats so much it will not be fit to hunt for the rest of the day. A full crop is a useless bird.

Hunger is not a positive experience, and the quadrant used to teach falcons and hawks is known as negative reinforcement, not positive anything.

Jacob L'Etoile said...

I am a falconer, I know what I am talking about here. While hunger management is certainly important I have flown hawks FAT, that killed consistantly. People think we 'starve' the birds into submission but I assure we don't. We ask them to work very hard, and keep them very fit. A starving bird is no fun to watch and dangerous to the falconer. A bird that is not starving, but too hungry is no fun either. They sit in a tree and sulk. There is very little punishment in falconry and the only reason I say very little is that I have heard inklings from people who fly them about some things you can do that may be described as punishment for golden eagles and harris hawks but these are very social, for raptors anyway. Other than that there is no punishment, none ever. And yes, food to a hungry man is a reward. Withholding food for an animal is not a negative reinforcement, or a punishment. There are limits on what you can train them to do, and I won't argue there are not. You cant call them off something, for example. I wouldn't take my dog out only trained as well as my birds. But it is a several thousand year old purely positive example of training a high drive predator to work free. It clearly shows what CAN be done, as well as what cant.

PBurns said...

The fact that you have flown a bird "fat" means you have taken a risk, and not necessarily a very smart one.

And, to put a line underneath it, "fat" here means that the bird actually ate a full meal at least once a day for the last three days or so.

As you know, the term for the "sharp" hunger that a hawk or falcon is kept in during hunting season is called YARAK. It has a name and it's in the dictionary such is the importance of the idea. Hunger is very much what controls and "trains" a bird of prey. See >> https://en.wiktionary.org/wiki/yarak

"Trained" in the context of falconry is perhaps a bit of an overstatement. It's more of a controlled release of a natural prey drive using hunger that is rarely fully abated when in hunting season. There's a reason very expensive digital scales are sold to falconers -- weight is everything if you want that bird to return.

Sure you can fly a bird "fat" i.e. a few grams higher than normal, but the bird is not actually "fat," and all it takes is ONE loss, and the game is over for the falconer and the bird too.... and probably forever for the bird who may get hung on its jesses if it goes feral or is not retrapped. See https://www.mikesfalconry.com/Scales-s/72.htm for scale prices. These scales are not being sold because the birds are not being weighed routinely, or because sharp hunger is not key to the "training" which is almost always in place (hunger).

Falcons and hawks are not birds trained to do tricks like pigeons or chickens or parrots. In fact, many birds are so poorly "trained" that they routinely foot their owners and batt off the hand in panic. These are birds with one pirmary "trick" -- return to glove -- and mostly return to glove due to "training" the bird that a return to glove is a quick way to end the "sharp" hunger inside.
It should also be said that some falconers and hawkers have such poorly trained birds that these folks can be said to be little more than "pageantry" falconers who never let their birds off a creance (aka a leash for falcons and hawks). I remember one falconer I know who went to a meet in the UK and out of a field of falconer, only he would let his bird fly free!

And of course, the birds are so apt to fly away when a meal is inside them that vast sums of money are spent on radio telemetry systems. See here for those prices. >> http://www.mikesfalconry.com/Radio-Telemetry-s/2.htm?searching=Y&sort=7&cat=2&show=150&page=1

Not all the birds are alike, of course. Its generally easier to fly a Harris Hawk -- a social bird that hunts in packs -- than a solo hunter like a Gos or a Peregrine. And, of course, an imprint bird hatched out of the egg and bonded to you sexually is another thing again. But is this "training"? It is not.

PBurns said...

Here's a quick link, and readers can find a hundred more with Google using "hunger" and "falconry: as their search terms. But from >> http://thefalconryschool.com/about-falconry/falconry-beginners-tips.html
_ __ _ _ _

"The hawk must be a little hungry when flown, as it ONLY returns to the falconer for food. It is weighed at the beginning of the training, (and of course every day afterwards) and the falconer will expect to reduce the fat weight of the hawk by about 10% to start with, and will adjust this 'flying weight' a little if necessary. It is also important to know that 'flying weight' is not a fixed figure, and will need to be adjusted upwards as the hawk's muscle mass, fitness and 'manning' level develops."
_ _ _ _ _ __ _ _

A thought experiment... Imagine a hard working native American of the 18th Century. No fat on him, as he lives by bow and arrow and spends his days in the wild where nothing is free. He is ripped and without an ounce of fat. Imagine we catch him, put him a handcuffs tied to a rope, and reduce his weight from his already thin 170 pounds to 153 pounds. Is he clinically starving? No, but he is DAMN HUNGRY. He is ravenous. His thin frame is 10 percent down and every part of his brain is screaming that point to him. Now we give him a bow and arrow and whenever we see a rabbit, we let him off the rope, but as soon as he kills a rabbit, we swoop in and wave him off, take the rabbit, and give him a small cookie instead.

Are we engaged in "positive" training? Not if you are that poor native!

Right there is exactly how we "train" hawks and falcons.

And what happens if our native gets five good meals in him and gets close enough to the woods that he looks like he might be able to sneak off? Well, we have radio telemetry on him for that. It will be hard for him to hunt on this land that he does not know, so we are reasonably sure we'll catch him again!

Actually, this would make for great futuristic short story; especially the part where the men who are "manning their natives" drink mead and eat mutton while they talk about how they are "training" their natives using "pure positive" reinforcement. The men, of course, would be chained to the wall a short distance from the fire, looking at their captors with ravenous feral eyes, not understanding a single word of the language, while never taking their eyes off the meat.

Jacob L'Etoile said...

Yes, the hunger is important but who ever said it wasn't. I seem to remember you writing several times to hold back a meal or two from your dog to make them more responsive to the reward your offering. Hunger is the natural state of a predator. They are ALWAYS hungry to some degree, excepting perhaps immediately after a meal. I encourage you to trap a nice big hen redtail, manage her to within a tenth of a gram so she is nice and hungry. Tell me how that works for you. My redtails usually have an ounce or so window they will hunt well in and after a year or so they hunt every other day almost regardless of weight. You can judge them almost purely by behaivour. Heck, they hunt in the wild as fat as they want to be. The thing you manage with a hawk is FEAR and hunger is one tool to do it with. Hawks were hunted for thousands of years without scales or telemetry. my scale cost $20 and I never fly with telemetry. As you know Google is no substitute for experience. One of the mistakes experience would have kept you from making is assuming that we got our birds hungry by holding back food. Which we sort of do, but no more so than you do with your dogs to keep them fit. Or do you feed them all they want whenevervthey want it? Much more important is keeping their system moving and we do that by feeding them full crops the same time each feeding and keep them active. We adjust the calories they get by feeding richer or poorer foods. After the initial training window my birds never go into their box after a hunt without a full or 4/5th full crop. The bottom line is falconers have been training high drive predators using reward only based training for thousands of years.

PBurns said...

The difference between boosting a dog's focus for trick training purposes and keeping a hawk always hungry to get a simple recall is that one is done episodically, while the other is chronic or ongoing.

Keeping a hawk hungry is much closer to using a a choke chain on full choke, which is relieved only when the animal sits. That's not reward-based training, but negative reinforcement. To be clear, it work, but it's not training in the conventional use of that term nor is it positive (though it feels good to feed a hungry animal).