FAQ
It is currently Wed Jun 28, 2017 10:35 am


Author Message
Pong
Post  Post subject: Help design experiments  |  Posted: Tue May 31, 2016 3:01 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
I lack rats and funding, so this may never get off the ground. Still I'd like to outline some experiments that, even as thought-experiments, may bring certain questions into sharp focus for prediction, explanation, debate.

What I'm wondering is: Just how does an omnivore like H. sapiens maintain nutritional homeostasis without external guidance? No single food provides all essential nutrients, and the choices are many, yet we somehow choose foods so that deficiencies are minor if not brief. Else we die. If the poor example of some modern consumers is too much, then consider the lab rat Rattus norvegicus - the question is the same. How does it know what to eat? Clearly omnivores can't go by instincts that they should eat, say, banana for one nutrient and tomato for another. And for humans at least we really can't discern by taste or appearance which nutrients a food contains. So apparently the nutritional properties of foods are learned by experience. It's this learning from experience I want to investigate with a series of experiments.

I'm not asking here if my question is smart or stupid, nor asking for citation of prior work. In any case I want to try it again, perhaps by more elegant and pointed experiments. So I'm asking for help designing experiments. I have no practice in this.



Firstly I guess I should check if nutritional experience really does affect future food choices. That'd be experiment #1. And being an honest scientist I'm trying to prove that no, wrong Pong, it doesn't.



As something of a thought-experiment at this stage I might as well use children for subjects. I can feed them daily bowls of cereal containing different nutrients. Say I supplement some breakfasts with the mineral iron, while others contain no iron but look and taste the same. The subjects are "blind". Slight deficiency in iron is common among children, and it's unlikely my little subjects would acquire unknown doses of iron from secret sources.

Umm... run it all over three weeks? I doubt I would cripple children with anemia in that period. I can't see how to design the experiment without making some groups moderately deficient in iron for at least a few days. I mean within the range that half their peers outside the experiment go though occasionally.

How to group the kids? Assume roughly 100 kids are available. Assume they've been screened and for now assume they're 10 years old all same age, same sex, same everything.

I could incorporate the iron as little candies in the cereal. The candy flavour would mask whether it contained iron, so children are still "blind". Also the candies could be coloured differently, so after some weeks one might allow the child to choose e.g. both red-candy and blue-candy cereals are placed on the table: "Pick one". If my basic hypothesis is wrong, then subtle tweaking of the kids' iron levels won't inform their choices. Right?

I don't know how to structure this experiment.


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Tue May 31, 2016 8:05 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
Not quite sure if this is off-topic, but here it goes.

I like the slight bitter taste of spinach, but I do know that others do not for the same reason. Is the preference culturally inherited, genetically inherited or a combination of both? Personal preference for certain foods may also be formed associatively - such as associating the bitter taste of medication at a very young age with that of other nutritional foods such as spinach. But alcohol generally is regarded as possessing a bitter taste for someone tasting it for the first time. Some developed a dislike for alcohol, but many developed a liking for it - alcoholism. One may dislike spinach but like alcohol for the intoxicating effect.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Tue May 31, 2016 9:10 pm
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
I believe your structure of hte experiment is too simple to be useful.

Clearly, what we eat is dictated based on what's available to us, something which differs with time, with seasons, and over our lives.

We tend to crave items we feel deficient in. Those cravings tend to come from learning experiences, what we ate when we grew up, what associations our brain and biology made upon eating those things. You'll never be able to control for what associations were formed to feed these cravings.

There's also the issue of our mood. If we're sad or down we make different food choices than if we're happy and up.
There's the issue of our activity level. If we're active we tend to crave more protein, or our blood sugar might get low after a heavy workout and we crave carbs.

This is an extremely complex issue and I'm sure there are people smarter than me who could help answer, but without funding, a firm awareness of existing research, rats, and other related facilities to test genes and metabolism, etc. I'm not sure you'll achieve the progress you envision.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 02, 2016 7:16 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Yep, the experiment has no structure! Yet. The thread is *really about* designing experiments. Strangely this science forum includes no methodology subforum. So I put this in Medicine & Health, with a Medicine & Health theme.

For sake of argument let's say I'm adding iron(II) sulfate to the breakfast cereal candies. And say absent that fortification subjects will grow kinda anemic after a few days. Enough to feel a bit tired. This condition is actually quite common and the reason you'll see many breakfast cereals include iron.

How about this:

Where we're dosing kids with iron let's call it +Fe. Withheld iron we'll call -Fe. Experiment #1 does not involve placebo, but it does offer several perceptions of the food: red candies or blue candies. Call that Red and Blue. So cereal with iron-fortified red candies would be +Fe Red. "Candies" would be much as the marshmallows in Lucky Charms cereal, or such.

4 groups of 25.
Each group has diets managed over three weeks.
Weeks called 1st, 2nd, 3rd.

Group A gets 1st +Fe Red, 2nd -Fe Blue, 3rd +Fe Red.
Group B gets 1st -Fe Red, 2nd +Fe Blue, 3rd -Fe Red.
Group C gets 1st +Fe Blue, 2nd -Fe Red, 3rd +Fe Blue.
Group D gets 1st -Fe Blue, 2nd +Fe Red, 3rd -Fe Blue.

The idea here is to provide time for subjects to associate feeling less or more healthy, with the colour of cereal candies. We also want to cancel innate colour preferences (children might find red candies more appealing than blue).

After 3rd week all subjects are offered a final bowl of cereal, their choice: Red or Blue.

The idea is more subjects will choose whichever cereal had the iron. Whether conscious or unconscious that would indicate people can detect nutrients in foods. But the data might be more interesting. For example we might find groups ending 3rd week with +Fe, more likely to seek variety because they don't really need the iron that day. I don't know what the results will be.

How should I improve this experiment #1?


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 02, 2016 3:53 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
I'm guessing you'll need to screen for participants who had similar dietary patterns for an extended period of time - regardless of their family and health backgrounds, for control purposes. Otherwise, the participants may be gaining the nutrients from other dietary sources from different meals at different times of the day - skewing the results. You will also want to screen for any inherent health conditions (liver?) that may affect the metabolism rates regarding Iron (hypoferremia or haemochromatosis). Orphanages would be the best place to look since they fulfill many of the conditions needed with additional screening.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 02, 2016 10:12 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Good points.

How's this:

Admit only children who already eat a bowl of cereal each morning. Then we questionnaire the parents about other dietary patterns and health issues. Finally we split our cereal-eater kids into two initial groups, by whether the cereal they already eat is iron fortified. That'll double the experimental groups to eight.

How much screening is really necessary? I'd thought that larger samples reduce the "noise" of unwanted variables. By the same token each time I divide my initial sample of 100, to isolate some variable, the now smaller group is even more skewed by yet another variable, so I must divide again...

Is there an accepted large-enough sample size for this sort of experiment?


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 03, 2016 12:43 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
You still have too many confounding variables. You need to better control for items unrelated to your study focus.

By comparison, it's like you're asking whether boxers or briefs are warmer without bothering to pay attention to whether the individual is wearing long sleeves or short, a jacket, a parka, silk or goose down, thermals, hats, gloves or other.

You'll never be able to accurately measure the impact of the underwear until you ensure everyone is each wearing the same thing in the same environment and the ONLY thing that's different between them is their underwear selection (an even better option is to have them otherwise nude).

For your food study, feed everyone two equal scoops of cereal at every meal. The ONLY thing that varies is the cereal type (amounts are the same here, too). Take blood draws before, during, and after. Compare and average results one day, one week, one month, and one year after you begin.

Then MAYBE you'll find something noteworthy, but you've still failed to control for genetic predisposition, environmental exposures, lifestyle, etc and your results will still be discarded as speculative and too imprecise to trust (though a sample size in the hundreds or thousands would obviously help minimize that noise).

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 03, 2016 8:17 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Okay, let's get sidetracked by the confounding variables problem. I agree it's important.

Take the question of whether boxers or briefs are warmer. It seems there are two divergent ways to proceed.

The first way, suggested in posts above, is to winnow down the larger sample of all persons wearing underwear, to just those who have everything in common. Of course the sample size shrinks, because we discard everybody who doesn't weigh exactly alike, and so forth... And yet we've "still failed to control for genetic predisposition, environmental exposures, lifestyle, etc" and more yet "still be discarded as speculative and too imprecise". In other words the way of removing all confounding variables is futile. We can't keep shrinking the sample size for every variable imaginable until... we're left with one individual.

Okay iNow. Point taken.

The other way, I suggest, is to use a large enough sample that the noise of confounding variables evens out across groups. So in our boxers vs. briefs experiment we should ignore whether subjects are male or female - it doesn't matter because males and females are distributed evenly enough across the two groups. We rather don't exclude people by sex, because that would shrink our sample such that body-weight differences becomes too noisy; likewise we don't exclude by body-weight because that increases the noise of different sexes. This way is the reverse of the former. Does this work?


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 03, 2016 3:40 pm
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
It'd help, but I'd still be reluctant to see your findings as relevant since you hadn't controlled for the countless other things. We need to be very precise to ensure we're measuring an actual effect and not getting fooled by something else happening... something else we failed to account for in our setup.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 03, 2016 7:30 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
There are indeed too many unaccounted variables to take into consideration when designing a experiment to yield useful data and preliminary conclusions that may warrant further studies. This is more pronounced with you design an experiment around iron absorption through the delivery system of cereal intake that is commonly taken with milk - which is known to inhibit iron absorption if I'm not mistaken.

I would suggest looking up published studies regarding iron absorption inhibitors in common dietary patterns on Google Scholar as a place to start. Look into how those studies were designed and conducted, what variables were taken into account, and other studies were referenced.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 03, 2016 10:54 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
iNow wrote:
you hadn't controlled for the countless other things.

scoobydoo1 wrote:
too many unaccounted variables to take into consideration when designing a experiment to yield useful data

In other words, experiments are futile.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 12:29 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
Imprecise uncontrolled experiments are rather often futile, yes, but not all experiments.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 6:51 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
All I'm hearing amounts to "You can't do credible experiments, Pong."

Okay... Can you outline a boxers vs. briefs experiment using human subjects, that should produce meaningful results?


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 11:43 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
Do you mean specific to iron uptake from cereal, or actual temp variations between boxers and briefs?

Side note: There is no animus here. It's just that science is hard and good science is extremely precise. The feedback above is on ideas and best approach, not you as a person. I hope it's not feeling personal as that's most certainly not the intent.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 1:32 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Boxers vs. briefs is fine... my experiment isn't for testing iron uptake...

My suggested preliminary experiment is meant to find any interaction between two variables. It doesn't want a control group. This in contrast to the single variable experiment commonly used in say testing a new drug ...or iron supplement... that must have a control and is really hard to block out unwanted variables.

I think you could work boxers vs. briefs into a factorial experiment.


Top
Lynx_Fox
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 2:18 pm

Joined: Wed Dec 30, 2015 5:17 am
Posts: 218
Location: US Pacific NorthWest

Offline
Are there any studies that demonstrate nutrient cravings already?

From all the perceived shortfalls in expert nutritional items items such as official claims of most people being low in chronically low in vitamin D, it seems at least for some nutrients, there's probably no cravings unless or until their body can no longer compensate for the low supply.


Top
Robittybob1
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 04, 2016 6:58 pm

Joined: Thu Jan 29, 2015 4:11 pm
Posts: 183

Offline
Lynx_Fox wrote:
Are there any studies that demonstrate nutrient cravings already? ....


Cattle short of phosphorus will eat bones. Last year I saw evidence of calves eating soil in a case of suspected mineral deficiency (cobalt?).


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sun Jun 05, 2016 2:36 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Lynx_Fox wrote:
Are there any studies that demonstrate nutrient cravings already?

From all the perceived shortfalls in expert nutritional items items such as official claims of most people being low in chronically low in vitamin D, it seems at least for some nutrients, there's probably no cravings unless or until their body can no longer compensate for the low supply.

Robittybob1 wrote:
Cattle short of phosphorus will eat bones...


There's a lot of cultural knowledge and anecdote, but very little scientific research to the point. All I've managed to rustle up are overly clinical tests, e.g. having subjects sniff synthetic banana and then measuring volume of banana milkshake consumed... this sadly lacks ecological validity, unless you're designing a new fast food product, and doesn't really address the mystery of real-world nutrient homeostasis. It seems the current theory is that if we can't taste or smell a nutrient, then we simply can't detect it, so we can't regulate it. By that theory cattle eating bones for phosphorus is unfounded because cattle can't taste phosphorus. Also it says, humans *can* regulate sodium because we can taste "salty" but humans can't regulate, say, potassium. This theory self-validates in a highly simplified lab setting, so it kinda passes for now. I think we could do better.

Anecdotally I could point out the "Snowbird" phenomenon, whereby Canadians migrate in large flocks to Caribbean resorts in winter, for the main purpose of sunbathing. They return north feeling recharged. It's the vitamin D of course. But they needn't taste the vitamin D nor understand the true reason they crave a suntan. I've actually tried to explain D production to regular Snowbirds, and they're like, "Well, whatever. I go because the beaches lift me out of the winter blues." On the other hand science is no more cogent about this: it's not supposed to happen.

It's that intuitive regulation of nutrients I want to investigate.

Besides that I also want to learn better experimental design. Replicating experiments, or trying a theory from different angles, is good science. Merely digesting and regurgitating prior work, IMHO, is not really doing science. *This* thread is really more about methodology, under a pretext topic to keep it down to Earth.


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Sun Jun 05, 2016 7:19 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
Pong wrote:
It's that intuitive regulation of nutrients I want to investigate.

Perhaps I'm mistaken, but I'm not entirely sure if there is an intuitive sense involved. The disease 'scurvy' is a good example. The 'cure' had to be prescribed rather than intuitively yearned by sufferers in early history. Availability is also a factor if we take into account the cultural cuisines available to those in maritime professions.

* I apologise for the lateness in replies, more detailed posts, and shortages of reference links, but it isn't the easiest task to do so while travelling abroad.


Top
Robittybob1
Post  Post subject: Re: Help design experiments  |  Posted: Mon Jun 06, 2016 12:06 am

Joined: Thu Jan 29, 2015 4:11 pm
Posts: 183

Offline
https://extension.usu.edu/behave/htm/pa ... /minerals/
Quote:
So, if you’ve noticed your animals eating soil, bones, feces or other unusual items, they may have a mineral imbalance. Can animals detect mineral deficiencies? Recent research says yes.


In that picture you can see they are eating subsoil at fair depth. IMO this could be indicating a cobalt deficiency.

May not be related to taste or smell but an instinctive behaviour.
"Mineral Nutrition: Are Animals Nutritionally Wise?" https://extension.usu.edu/behave/files/ ... 20Nutr.pdf

http://www.mla.com.au/CustomControls/Pa ... Tnt3BqiA== on page 9 are photos of cattle eating bones in phosphorus deficient areas of Australia.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Mon Jun 06, 2016 10:54 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Thanks a ton Robittybob1. Those links you dug are bang on what I'm talking about. I should'a guessed insight and experiment would begin with livestock.

scoobydoo1 wrote:
I'm not entirely sure if there is an intuitive sense involved. The disease 'scurvy' is a good example. The 'cure' had to be prescribed...

Scurvy's a bad example. Scurvy was caused by dietary deficiencies enforced by sea rationing. Scurvy's a good example of unintended experiment. Ships served as quarantined laboratories complete with well-documented compulsory diets and no way subjects could eat better.

Anyway we could argue forever whether there's sufficient evidence to support my claim... but I'm not trying to assert a claim here. I'm trying to test it. I'm quite happy to say there is no prior work supporting this.

I supposed iron would make a good nutrient variable. Is there a better mineral or vitamin? For ethical reasons I want to reassure parents that... hm, how to put this... "Over the course of the experiment your child will receive ordinary mineral supplements, added to the cereal, no more than is added to popular breakfast cereals." Then let parents guess we're just testing colour preferences. The parents must be blind too.


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Mon Jun 06, 2016 6:05 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
Pong wrote:
scoobydoo1 wrote:
I'm not entirely sure if there is an intuitive sense involved. The disease 'scurvy' is a good example. The 'cure' had to be prescribed...

Scurvy's a bad example. Scurvy was caused by dietary deficiencies enforced by sea rationing...

The early records of scurvy are predominantly surrounding those in maritime professions, Yes.

But there are relatively modern day examples of scurvy occuring as well, and the sufferers aren't in maritime professions. You can refer to the linked list below and browse for entries detailing occurances of 'scurvy' due to dietary preferences.

http://www.pubpdf.com/pub/6476860/Scurv ... ary-habits

If the onset of 'scurvy' is due to a specific nutrient deficient diet, the immediate remedy ought to be to 'intiutively' yearn for and seek out foods that possess the missing nutrients to maintain 'nutritional homeostasis' as it is called. But the sufferers mentioned in the links did not do so eventhough they could have.

In cases where dietary patterns are limited due to the lack of nutrient rich foods, I might concede to your point. But since there are occurances where it's simply a matter of personal dietary preference, I am unable to.

To be frank, I had originally thought that we did indeed have some form of intuitive sense to yearn for and seek out nutrient rich foods to maintain our health, but so far, what I've uncovered regarding scurvy doesn't allow me to hold on to that original opinion.

* Apologies for not adding more specific links to the post. Submitting a thread post, copy and pasting links, amending quotes on a smartphone while travelling abroad isn't as easy as doing so on a desktop computer back home.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 1:31 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
*Sigh* Insane sidetrack...

Scoobydoo1, I do appreciate you're wanting to contribute, and I know your intentions are good. But please get out of this forums mode of searching for debatable statements and drawing authoritative evidence against. I don't wish to debate. I don't wish to enter a regurgitation contest. I wish to test. Test as though we don't already know the answer.

Earlier you said "too many unaccounted variables to take into consideration when designing a experiment to yield useful data". Could we at least continue along that line?

How should I design my experiment to get around that problem?


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 1:47 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
Simple. Control for extraneous/confounding variables. Start small, build up on success.

Your question is quite broad and valid replies have been offered. What about the replies you've received do you find lacking?

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 2:32 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Your answer is also quite broad. :lol:

Can you apply it specifically to "Experiment #1" as I've developed thus far?

#1 is meant to test subjects for any association between a hidden nutrient and a superficial difference in food presentation. Those are the two intended variables.
Pong wrote:
Where we're dosing kids with iron let's call it +Fe. Withheld iron we'll call -Fe. Experiment #1 does not involve placebo, but it does offer several perceptions of the food: red candies or blue candies. Call that Red and Blue. So cereal with iron-fortified red candies would be +Fe Red. "Candies" would be much as the marshmallows in Lucky Charms cereal, or such.

4 groups of 25.
Each group has diets managed over three weeks.
Weeks called 1st, 2nd, 3rd.

Group A gets 1st +Fe Red, 2nd -Fe Blue, 3rd +Fe Red.
Group B gets 1st -Fe Red, 2nd +Fe Blue, 3rd -Fe Red.
Group C gets 1st +Fe Blue, 2nd -Fe Red, 3rd +Fe Blue.
Group D gets 1st -Fe Blue, 2nd +Fe Red, 3rd -Fe Blue.

The idea here is to provide time for subjects to associate feeling less or more healthy, with the colour of cereal candies. We also want to cancel innate colour preferences (children might find red candies more appealing than blue).

After 3rd week all subjects are offered a final bowl of cereal, their choice: Red or Blue.

Perhaps a significant number of subjects will choose a colour associated with iron.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 3:10 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
How are participants selected? How are you controlling population and their relevant attributes?

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 7:03 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
iNow wrote:
How are participants selected? How are you controlling population and their relevant attributes?

As said before, I'm no expert at designing experiments. Well, I'm winging it. Please guide or correct.

Selection of subjects, controlling attributes:

By how the experiment is structured, it doesn't need a classic control group. This is a factorial experiment, where opposing groups act as controls for each other. So it shouldn't matter if, say by chance, half the subjects are vegan, so long as vegans are equally distributed across the groups. Then I think I may be pretty loose in how I gather and screen subjects, so long as they get randomly assigned into groups of sufficient size to smooth out the noise of individual differences.

I'm doubtful 4 groups of 25 is really enough, but I picked that as a starting point. On the other hand recall the historic breakthrough experiment on scurvy included only two men in each group... two men fed lemon for a few days only... and yet results were accepted as scientifically valid. Strong results with groups of 25 would warrant repeat experiment with larger groups. I need to consider the cost of doing this tentative trial. Such is life.

Subjects are children, so invitation through the parents. Place ads in local schools, daycares, parent magazines, etc. We want to avoid collusion between subjects (and families), so one child per family only, and "How did you learn of this experiment?" to exclude people who know other participants. Maybe then draw cards to get a random spread across the city.

I'm quite aware some children will enter the experiment with varying degrees of iron deficiency (anemia), as it's pretty common among kids. Also some will have had daily iron supplement through their normal, fortified, breakfast cereal. But *I think* randomized assignment will equalize these differences across the groups. In other words confounding variables will be distributed across the groups, so they won't significantly affect comparison of results between groups. If that's true, then good intentions to manually select and assign subjects case-by-case is actually a bad idea, because that would be tampering. The less I tamper with the sample, the better. No?


Top
Rory
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 12:34 pm
User avatar

Joined: Wed May 07, 2014 6:02 am
Posts: 1902

Offline
Quote:
I lack rats and funding


I like this... I'm going to put it on my business card

_________________
If you are doomed to be boring - make it short. Andre Geim


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 6:29 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
Pong wrote:
How should I design my experiment to get around that problem?

Select for participants with highly similar (regimental even) active/inactive lifestyles where their diets and daily activities do not differ too greatly. A general clean bull of health may or may not be preferred when testing for an `intuitive sense`, since we know not whether the sense expresses itself all the time, or only when the body is currently deficient in specific nutrients. Ideal candidates/participants that fulfill these criterias can be found in,


1. Orphanages, they have and will had highly similar dietary patterns over an extended period of time. For youngsters both male and female.

2. Military servicemen/woman, for the same reasons, but with added difference age groups and preexisting health screenings. For adults both male and female.

3. Prison inmates, may or may not be ideal too, depending on any prior/existing substance abuse and aftereffects. For adults both male and female.


* Will add a later post with recommended improvements to design parameters for nutrient delivery system, and such. (Posting on the go here)


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 7:27 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
If the preferred nutrient delivery system is through oral ingestion, you may want to consider whether it is fed using cooked/processed or uncooked/unprocessed/raw ingredients. There is a wide variety of food choices to be considered for both cooked and uncooked ones to choose from depending on the nutrient you have decided to test.

The humble potato is one such candidate. It may prepared steamed, baked, boiled, etc. It contains varying levels of different nutrients such as potassium, iron, vitamins C, etc. The big plus may be that it can be very palatable to a wide range of people depending on how it is prepared, and yet taste mildly bland without any condiments to accompany it (testing for gustatory & olfactory senses). Accompanying condiments may include those that promotes/inhibits nutrient absorption as a means to add an additional layer to the experiment.

* Have to stop here for today. Catching an early flight to Croatia - Zagreb in the morning.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Tue Jun 07, 2016 11:40 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
scoobydoo1 wrote:
Select for participants with highly similar...

I still don't understand how that helps. Selection means I'm doctoring my sample.

For example say I'm testing whether one obscure variable affects American voting choice, Democrat or Republican. Then if I admit only prison inmates, or people in the military, or those in the same tax bracket or whatever... how does that remove confounding variables? Seems rather my screening the sample introduces confounding variables. Why not simply take a random sample from all Americans in all their variability?


I figure breakfast cereal a good delivery, because most kids are content to eat that every day of the week at roughly the same hour. It's the most routine part of their diet. I can't realistically control all other aspects of a child's diet, and moreover I shouldn't because that would produce results only valid within the unnatural parameters of my experiment. I want results that are valid in the real world.

Incidentally, did you know that the first modern breakfast cereal, Kellogg's Corn Flakes, was invented for use in lunatic asylums? Stranger yet it was supposed to discourage masturbation.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 12:11 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
It's about ensuring your sample allows you to focus on your studied variable. You're controlling for confounding variables by maximizing the homogeneity of your population.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 2:57 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
You really distrust randomized controls don't you?

The thing is, I don't trust myself to screen without biasing the sample, maybe screwing the experiment completely. You shouldn't trust my results if I doctored the sample beforehand. And for the purpose of this first experiment I see no need for it. Even if I do screen and get interesting results, others must then take my screening into account... just more work and complication for everybody.

Here's a reason *not* to use only similar individuals:
scoobydoo1 wrote:
A general clean bull of health may or may not be preferred when testing for an `intuitive sense`, since we know not whether the sense expresses itself all the time, or only when the body is currently deficient in specific nutrients.
...and there could be other hidden variables I'd never imagine.

So I propose to solve that problem scoobydoo1 brought up, by allowing any subjects, healthy or unhealthy, into the experiment. And to attain the homogeneity you desire, iNow, I randomly assign subjects across all groups regardless of their health. In other words I'm willfully blind to individual differences. I understand that's the opposite of screening for sameness.

It works doesn't it?


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 3:04 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
It's not about trust. It's about training and learning about the easiest ways to allow your experiment to unnecessarily fail, and how to proactively avoid those easily avoidable pitfalls.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 5:15 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Yeah pitfalls are a drag for everybody. But we learn as we go and stuff happens.

I'm in no hurry to draft these experiments so perhaps revisit the topic later, unless scoobydoo1 has more to add.


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 6:55 am
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
Pong wrote:
I still don't understand how that helps. Selection means I'm doctoring my sample.

The unknown variables present are the absorption, storage, depletion, and expenditure of the nutrient(s). Having participants with a somewhat controlled/regimental - active/inactive lifestyles ranging from reduced physical exertion from lazing around all day or sitting in a classroom (such as students) to say having to train for various sports or being in a military boot camp conditions where you have increased physical exertions. Participants who are already living such a lifestyle for prolonged periods both before and after the experiment beings reduces the chances for statistical anomalies, and the need for any major disruption to their current lifestyles. This is valid for similar reasons where their highly similar dietary patterns is concerned, and to reduce the chances of them having additional intake of nutrients during different meals (or skipping meals) and intake of foods that may even promote/inhibit the absorption of said nutrients. Having participants with wide ranging lifestyles with different nutritional needs and/or say daily 'unhealthy' dietary patterns within the group in the initial phase of the experiment just isn't sound in the beginning stage. You may incorporate that into later phases once you've gathered some useful data from somewhat controlled conditions.

Pong wrote:
I figure breakfast cereal a good delivery, because most kids are content to eat that every day of the week at roughly the same hour. It's the most routine part of their diet.

A delivery system that requires ingesting a food choice with the accompanying milk that is known to inhibit say iron absorption would not be wise. Besides, if their wide ranging lifestyles and dietary patterns allows them to ingest and absorb the nutrients you're using in your experiment during other meals, the yielded data will not be remotely useful in the analysis stage.

Pong wrote:
I can't realistically control all other aspects of a child's diet...

You wouldn't need to if you select for participants that are already and will be leading a 'regimental' type lifestyle. There wouldn't be any major disruption to their lifestyles. The experiment can be performed in various stages. You can do athletes, military serviceman/woman, kindergarten children, prison inmates, etc.

Pong wrote:
I want results that are valid in the real world.

It can, if you account for the many variables. Additional experiments with different testing phase will build up the amount of data you will eventually collect. Data analysis will bite you in the backside if there are too many unaccounted variables to begin with.


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 12:09 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Scoobydoo, many thanks for addressing my experiment in detail. :)

scoobydoo1 wrote:
A delivery system that requires ingesting a food choice with the accompanying milk that is known to inhibit say iron absorption would not be wise.

Excellent point. I doubt so many cereals today would be iron fortified if this was a significant problem with iron, but I'll look into. In any case I was planning the same type and volume of milk be given all subjects. That's easy to regulate. Unfortunately the (cow) milk and type of grain will make a small percentage ineligible.

Sugary kids' cereal with milk I imagine is the best... er, "delivery system". Bet iNow will corroborate that as he has a little one of his own.

BTW to be clear: subjects don't have any choice about their breakfast cereal, until the final bowl after the 3rd week. Which bowl they choose then, is the output variable.

scoobydoo1 wrote:
Having participants with wide ranging lifestyles with different nutritional needs and/or say daily 'unhealthy' dietary patterns within the group in the initial phase of the experiment just isn't sound in the beginning stage.

This has been said before. Your reasoning, I know, was that tight selection is necessary to eliminate individual differences between subjects. You believed the only way to deal with variation is to screen and regiment until all subjects remaining are practically alike. Well, I've got good news for you. There's a better way, sometimes quoted as "the gold standard" in this sort of experiment, and it's much simpler and easier! What is a randomized controlled trial in medical research? Put simply: we distribute the various quirks of individuals throughout the groups, so that statistically those quirks nearly cancel out.

For example if we randomize 100 individuals of various ages into two groups, we'll find the average age in each group nearly identical. In this way you needn't manually select only, say, 35-year-olds for your study. You needn't select for sameness. Not only is this logistically easier, it diminishes the power of some unknown/unwanted variable unique to 35-year-olds. Supposing 35-year-olds do carry some unwanted variable, and you blindly include them. Well, they're a drop in the bucket of your sample, so they won't spoil your experiment. And if 10 of 100 subjects are that age, it's unlikely all 10 will land in the same group. In any case it's better than a study where you selected 35-year-olds exclusively, and thus introduced a powerfully confounding variable.

Randomized samples can take multiple weird variables in stride. For example you can test a mixed bag of jellybeans including all sorts of flavours and colours - confounding variables - without a problem, if you shake them up and deal randomly into large enough groups. You can imagine how that looks, and it's okay: the variations cancel out. This is totally valid if your goal is to test "jellybeans". Not "pink jellybeans" or "banana jellybeans", or whichever jellybean you secretly suspect will give the result you want. Remember what I said about not trusting myself - I take it seriously.

Most examples of randomized controlled trials involve one group receiving a newfangled medical treatment, the other receiving null/placebo. And typically the subjects are selected, from people who might benefit from the treatment. E.g. they all have eczema. My experiment differs in that I want to test the general population; and I want to test for relationship between two variables. Not your standard RCT, but I'm enamored with the power of randomized group assignment to flatten extraneous variables, so unless you have an extraordinarily compelling reason not to, I'll go with that strategy.


Some screening is absolutely necessary, if only for logistic or ethical reasons:

No lactose intolerance.
No allergy to the cereal grain(s).
No serious medical conditions.

More caution:
I would ask which (if any) breakfast cereals the child has eaten for more than a week in the past year. Ask if any bad reactions or complaints about those cereals. Include only subjects confirmed to tolerate cereal roughly comparable to the test cereal. Of course this also greatly increases the chance of compliance.

Anti-collusion:
One child only per family.
Omit (at random) additional families living on the same block.
Ask if know anyone else in the study - pare those away.

Child's age:
Hmm. I dunno. Children often have inscrutable food preferences. There may be a developmental window one way or another, so I want to catch that and catch around it.
I want school attending children because they have a morning routine.
Older than 10-years find too much food and snacks outside the home.

What else about the subjects?

Likely a few can't distinguish between different coloured cereal-candies. Colourblind, diagnosed or not. I think the sample can absorb them, their influence will be minor. No, really I *should* include them, they're a part of the general population... what was that about selecting to produce the results I want?

What am I missing?


Top
scoobydoo1
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 3:55 pm
User avatar

Joined: Fri Sep 05, 2014 9:39 am
Posts: 199
Location: Singapore

Offline
I require clarification. What exactly is the objective of the experiment?

Is it to gather data on whether humans possess an 'intuitive sense' to yearn for and seek out nutrients when they possess deficient storages of, when they have ample storages of, or both at the same time? (I do recall you mentioning that this experiment is to be performed on the notion that we do not already know the answer.)

Or, something else entirely?

Pong wrote:
BTW to be clear: subjects don't have any choice about their breakfast cereal, until the final bowl after the 3rd week. Which bowl they choose then, is the output variable.

There are two major factors that will render the data from this experiment quite useless.

1A, Participants consuming sufficient nutrients (say iron content) outside of the breakfast meal (your portion of the experiment) may affect which 'bowl of cereal' they choose and why they have chosen it.

1B, Participants consuming insufficient nutrients (say iron content) outside of the breakfast meal (your portion of the experiment) may similarly affect which 'bowl of cereal' they choose and why they have chosen it.

2A, Participants consuming sufficient nutrients (say iron content) outside of the breakfast meal (your portion of the experiment) and living a lifestyle with low physical exertion - may affect which 'bowl of cereal' they choose and why they have chosen it.

2B, Participants consuming insufficient nutrients (say iron content) outside of the breakfast meal (your portion of the experiment) and living a lifestyle with high physical exertion - may similarly affect which 'bowl of cereal' they choose and why they have chosen it.

Pong wrote:
What am I missing?

At this point, I'm not entirely sure.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Wed Jun 08, 2016 5:27 pm
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
The concept being conveyed here, Pong, is similar to testing aerodynamics of a vehicle. To truly measure how it performs aerodynamically, you need to put it in a wind tunnel where you're controlling from errant gusts and flows, or even inconsistent wind speeds. You make your test conditions as ideal as possible to ensure what you're measuring is related to your actual objective. Simply putting the vehicle outside and testing aerodynamics there means your data will be skewed by other wind sources and various aero flows. You NEED the wind tunnel to do this properly. Like music, you can hear the specific notes you're listening for better in a sound room with quality headphones on than in a crowded football stadium during a game.

Same here with your cereal idea. You are measuring one very specific thing. Make sure your data addresses just that one thing and is not ruined by other factors. You NEED to control for those other items discussed above.

Similarly, let's say you were trying to test alcohol absorption. You decide to pour whiskey in specific amounts every evening at 5pm and run your test, but you don't bother to prevent your subjects from also drinking beer and wine and vodka during the rest of the day and you don't bother tracking how much of those things they're ingesting that could skew your findings. It's a rubbish test because you're failing to control things that will affect your study purpose. Apply this same logic to the cereal and you understand where we're coming from.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Lynx_Fox
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 09, 2016 4:28 am

Joined: Wed Dec 30, 2015 5:17 am
Posts: 218
Location: US Pacific NorthWest

Offline
I won't go as far as Inow, because the types of controls are almost NEVER practical for any complex natural system. His point about controlling as much as possible is certainly true though. In and absence of that, it takes a LOT of data and fairly complex statistical tools to tease out various relationships--and even then needs (like in all science) to be supported by a hypothesis that not only accounts for observations but also explains the possible relationships. All this certainly applied to nutrition, being a complex system, with severe ethical constraints --and hence the huge amounts of research into the topic, and occasional dubious conclusions (though the media and snake oil alternative med industry often gets the science reporting wrong).


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 09, 2016 2:09 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
iNow wrote:
similar to testing aerodynamics of a vehicle...in a wind tunnel

That's good for detailed study where we want to observe the action of something.

Suppose I merely want a true/false result i.e. significant/insignificant:

I wish to test for better performance of a modified aileron trim-tab for passenger jets. This modification is safe to fly - safe as eating Kellogg's Corn Flakes. I have 100 jets at my disposal, ready to equip the modified trim-tab in their daily flights, and log their fuel use over three weeks. And that's cheaper and less invasive on the airlines, than bringing carefully calibrated jets into windtunnels.

Yeah, all jets will be subject to countless confounding variables as they fly hither and thither across the country. Some fly through storms, some haul more weight, etc. They'll also generate a lot of real-world performance data. I may compare that against another 100 jets that flew without the modification on same routes over the same weeks. Do you really think it likely, if my modification makes a significant difference, that won't show up in the comparison? 'Cause all I'm looking for is a small fuel savings over hundreds of flight-hours: true or false.

Then if true, do you think the airline industry would doubt the real-world validity of my findings?

Maybe I should have made the purpose of my experiment #1 more clear. Seems you're demanding more of it than I intend.

Lynx_Fox wrote:
a LOT of data and fairly complex statistical tools to tease out various relationships

I'll repeat the one simple question posed with the experiment, shortly.
Lynx_Fox wrote:
nutrition, being a complex system, with severe ethical constraints

Providing parents with ordinary breakfast cereals, for their kids.
scoobydoo1 wrote:
I require clarification. What exactly is the objective of the experiment?

Lol. I almost forgot.
Answer:
Pong wrote:
apparently the nutritional properties of foods are learned by experience (Pong's hypothesis)

Firstly I guess I should check if nutritional experience really does affect future food choices. That'd be experiment #1.

..and then I began to outline an experiment that, by our current understanding, should yield no interaction between two variables: unconscious nutrient intake, and superficial food appearance.

The experiment includes 4 groups, each fed opposing schedules of nutrient or placebo, with red or blue visual cue in the food. I need 4 groups to rule out bias due to order of colour presented, or innate colour preference. After a 3 week schedule meant to condition subjects to associate a hidden nutrient with a colour cue - if such is possible - subjects are allowed to choose whichever colour they desire that final day. Their choice is the output variable.

If current theory is correct, the colour choice output should be statistically similar across the groups, since subjects had no way of learning to associate one colour or another with a nutrient.

I have no idea what a statistically significant result would be. The experiment is only meant to test an hypothesis, true or false. If false (results inconclusive, insignificant) then I've simply corroborated the current theory that humans can't intuitively associate foods with their nutrient content. If true, then I'll have interesting data to puzzle over.

scoobydoo1 wrote:
(Pong simplified)1, Participants consuming different levels of nutrients outside the experiment may affect (their output choice).

Yes. I believe a large enough sample can smooth those differences sufficiently for my purpose.
scoobydoo1 wrote:
(Pong brutally simplified)2, Outside nutrients and high or low physical exertion may affect (their output choice).

Yes. Again, large enough sample with randomized assignment to groups, should fairly distribute the differences. I accept there will be some noise remaining when we compare groups. If my hypothesis is good, it'll show plainly through such noise.

Heavy screening and subject monitoring is an ideal easier said than done. It requires a lot of work and I'm sure it's a drag for participants... some will drop out when they find the experiment more nuisance than novelty. Then we're entering the twisted landscape of eccentrics who like it or make a living getting compensated in proportion to the extremity of inconvenience, or who have no choice. Not a good representation of the population. Resources are better spent - I imagine - on increasing the sample size. In other words I can afford to conduct rigorous screenings and pay the babysitter for a very small sample, or for the same money I can distribute honor-system packages to many many more households.

If there is a "confounding variable" very common in the population, which diminishes the output to insignificance, that's okay: the experiment still works as intended. I don't consider the variable confounding. It's the way things really are, despite what I might like my experiment to indicate. Tough luck, Pong, that's science. I'm unwilling to fix conditions of this experiment to produce a "better" result.

iNow wrote:
let's say you were trying to test alcohol absorption. You decide to pour whiskey in specific amounts every evening at 5pm and run your test, but you don't bother to prevent your subjects from also drinking beer and wine and vodka during the rest of the day and you don't bother tracking how much of those things they're ingesting that could skew your findings.

In the above, iNow means to measure blood alcohol changes over time against volume consumed. He's not interested in surveying typical alcohol levels, nor whether his treatment results are externally valid. He does not want to know how people naturally consume alcohol. He wants a purely clinical, internally valid test, and he *can't* use subjects that reflect the normal population. In this case he may fix conditions to produce results that really are "better".

That is completely different than what I want to do.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Thu Jun 09, 2016 4:41 pm
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
Pong wrote:
I wish to test for better performance of a modified aileron trim-tab for passenger jets. This modification is safe to fly - safe as eating Kellogg's Corn Flakes. I have 100 jets at my disposal, ready to equip the modified trim-tab in their daily flights, and log their fuel use over three weeks. And that's cheaper and less invasive on the airlines, than bringing carefully calibrated jets into windtunnels.

Agree, that would be a valid test. You have 100 jets that are exactly the same. You have them flying the same routes every day. You monitor their fuel use for 3 weeks after making ONE single change. EVERYTHING else remains the same. That's a valid test.

Now, tell me how this applies to the different kids with different genetics and different diets and different behaviors and different schedules and all of the other things that are NOT the same in your proposed study population and methods?

Pong wrote:
Yeah, all jets will be subject to countless confounding variables as they fly hither and thither across the country. Some fly through storms, some haul more weight, etc.

Are you arguing that you should NOT at least attempt to control for weather conditions and weight? Not sure what point you're making. Of course those things should also be held constant (or as close to it as possible) when running your tests.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Lynx_Fox
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 10, 2016 8:48 pm

Joined: Wed Dec 30, 2015 5:17 am
Posts: 218
Location: US Pacific NorthWest

Offline
iNow wrote:
You have 100 jets that are exactly the same. You have them flying the same routes every day. You monitor their fuel use for 3 weeks after making ONE single change. EVERYTHING else remains the same. That's a valid test.


But of course their trips were never really ever the same...because the weather isn't the same from day to day...thus the plane operates under different wind conditions, different temperatures, different flight altitudes, different humidities, different flight patterns, and different passenger/cargo loads waiting to land or burn off fuel etc etc etc.

Strict controls almost NEVER exist or are possible outside of a lab. Even the simplest experiments get damn complicated in any natural environment.

But hell, science would be damned boring otherwise.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Fri Jun 10, 2016 11:07 pm
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
I totally agree. The larger point, of course, is that some variables CAN be controlled for, and ignoring that mandate unnecessarily defeats the purpose of ones experiment and stupidly limits the validity of the effort put forth.

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 11, 2016 2:23 am
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Lynx_Fox wrote:
Strict controls almost NEVER exist or are possible outside of a lab. Even the simplest experiments get damn complicated in any natural environment.

Totally agree with the first. As for the second, natural environment is not a problem to overcome if what your testing is the natural environment. For example if you want to count birds in your yard. You needn't exclude the sparrows from the chickadees. You needn't put out feeders. You shouldn't. You're observing the natural world, just as it is. And not only observe, you may place completely valid experiments directly in the natural world.

The natural world I want to test, is if peoples' everyday food choices are affected by the hidden nutrients in different foods. Do they intuit something?

It's like I want to test whether people know an oak from a beech. Well gee there are a lot of variables in that. One could run down a fool's path trying to select ideal subjects for clinically testing a single variable, say how controlled shades of green strike identical retinas of people screened for colour sensitivity. Or, one could just put the bloody question directly, to a fair sample of the population: Show random subjects two trees, ask "Which one is the oak?"

My point is, Lynx_Fox, the simplest experiments aren't always "damn complicated in any natural environment".


@iNow. This digression has reached the point I've got to wonder what's behind it. Excuse the intrusion...

I find your position ironic. My guess is, you've grown so used to consigning bad method and fallacy to the domain of non-science, that, once you've decided something is "science", the possibility it suffers the same flaws is unthinkable. A good scientist like yourself is incapable of selection bias, spotlight fallacy, cherry picking. Those are methods you've explicitly, repeatedly, insisted I employ to be properly scientific. In opposition to my gut reluctance and articulate pleas. It seems you're sure these methods are okay if done under the mantle of science.

I appreciate you're trying to help me get the results I want. You're treating me as a fellow scientist.

Now, where I'm coming from. I don't rate myself a scientist. It's what I do and how I do it, that is scientific, or not. When you said "It's not about trust" I let that slide. But that's really at the heart of the matter. I won't trust myself to do good science because I'm a scientist. That's my own position, equally ironic as yours.

Some posts ago I read "pitfall" as cue to provide you a dignified escape. Seems I read too much and this hole you've dug yourself iNow is where you'll stand anyway. Neither of us will change our positions. In fact I've become entrenched, thanks. If we're very lucky some outside mediator will show us how we're both right. 'Else, we should probably agree to disagree and move on with the experiment, granting Pong's silly aversion to selections that spotlight the variables he wants to pick out.


Top
iNow
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 11, 2016 9:18 am
User avatar
Original Member
Original Member

Joined: Thu Aug 04, 2011 11:40 pm
Posts: 5571
Location: Iowa

Offline
Largely trying to pass on to you many of the instructions I received from my instructors and mentors when setting up my own experiments.

Happy to bow out, though. No worries. Just keep in mind that a huge part of doing science and running experiments is about poking holes in ideas and finding flaws and helping to shore them up so any results are more robust and trustworthy.


Image

_________________
iNow

"[Time] is one of those concepts that is profoundly resistant to a simple definition." ~C. Sagan


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 11, 2016 12:17 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
You're gracious. And the challenge did force me to study up behind the scenes.

I've more to be wrong about...

1) Child subjects. Bad idea?

Using breakfast cereals seems most appropriate because this how most of our population gets dosed with supplemental nutrients already. So my additions should go unnoticed, plus I regulate the ...uh, confounding variables... of nutrition from a subject's regular cereal.

Children are the obvious consumers. Indeed I can't imagine any group more universally compliant with a particular food. Some children will eat little else. Parents have no reason to falsify a child's compliance, nor fake any other data regarding diet or whatever.

Adult subjects on the other hand, may balk at the sight of, say, blue marshmallows in their cereal. And adults are more able (and likely I think) to flout the experiment since they speak for themselves. I don't trust the self-monitoring, and self-reporting.

One possible flaw with including only children, is children's food preferences and nutrient sensitivities might be unlike adults'. Many children are extremely picky eaters, most commonly manifested by rejecting anything perceived as mixed (the understanding lately revolutionized elementary school cafeteria food). My hopeful speculation is this is actually their nutrient-identification system in development - they need to eat foods as discreet components to learn what each component does. But anyway the danger to my experiment is perhaps only adults would show a result.

Also possible developmental stages I might hit or miss, depending on the ages included. I reckon after about 10 years age parental control and monitoring of diet declines swiftly. By 13 most children do their own thing besides family dinners most evenings. Teens would be impossible subjects.


2) Unregulated experiment. :o

To maximize the sample, my experiment would be delivered as kit to selected households. Certainly the biggest bang for my buck.

I'd have to rely on parents to administer the cereal as directed. I can provide each measured serving in a baggy.
They'd have to buy the correct milk though, measure the correct amount (roughly). Each morning... roughly same time should be okay.

I shouldn't load them with work. Like doing a lot of monitoring, filling out reports. While I'd love to have the additional data, I'm afraid overtaxing participants will compromise things. They may tire of the experiment and drop out. They may find it easier to make up data.

Would having a small additional group, one that was monitored, allow me to see if the larger groups were failing in some way? This group would not contribute to experimental results. It's not exactly a control group. Is there a term for this?

How do I improve participant experience? I need parents and children to do this eagerly. Besides paying them in proportion to their inconvenience.

Perhaps I could buoy their participation via internet e.g. daily email, blog 'w feed of related content to make them feel consequential, website for optional data submission. While I need them blind to the exact nature of the experiment I could probably make them partial insiders. If it's really slick & appealing they might even pay for the privilege! Dumb idea?


Top
Pong
Post  Post subject: Re: Help design experiments  |  Posted: Sat Jun 11, 2016 11:54 pm
Original Member
Original Member

Joined: Sat May 28, 2016 2:53 am
Posts: 219

Offline
Sorry for spamming random ideas, I think I solved the participant engagement problem:


Participant Engagement

Each family is given a unique 3-digit alphanumeric password, that makes the first half of their logon to the experiment website. Families are engaged to comply and report, by a daily variable encoded in the delivery preparation, which they enter as the second half of their website logon. The daily variable is an alphanumeric string embedded in the cereal "candies". This employs the gimmick of pattern revealed when wet by milk, e.g. the pot of gold which magically appears when you pour milk on Lucky Charms. In my experiment each daily baggy of cereal contains a unique code that magically appears at time of serving.

Child subjects and parents will actively focus on the cereal of course. This ensures that children also see the overall colour of cereal candies, colour being the real key variable we want them aware of. Whether the child subject reads or thinks much about the apparently random string is irrelevant, and won't affect the experiment, as I'll explain shortly.

This is a good moment for parents to visit the site, while the subject child eats, and enter their unique 3-digit password plus whatever code appeared for that day. To their point of view, the code merely completes a logon password which unlocks some web content. They may be given the false impression their daily code is special to them, or random. However the daily code is identical for all participants - it only confirms which baggy (i.e. 1st, 5th, 20th) was served that day. All subjects who properly follow the regime schedule receive the same sequence of codes, the same web experience. Maybe after the parent fills in a few data fields, an entirely irrelevant game will pop up for the child's amusement.

I've seem similar schemes deployed with commercial products, but haven't looked into them. Perhaps my solution is not so new.


Top
Display posts from previous:  Sort by  
Print view

Who is online
Users browsing this forum: No registered users and 0 guests
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
Jump to:   


Delete all board cookies | The team | All times are UTC


This free forum is proudly hosted by ProphpBB | phpBB software | Report Abuse | Privacy