What to cite when the research frontier is moving fast…
One of the things I do in my economics classes is assign term paper assignments, which is not terribly common in undergraduate economics courses (problem sets and exams are more typical). For many students, this will constitute the first economics paper they’ve ever written. Yet this type of assignment requires that students become more discerning regarding the material they take seriously when doing their own research. It seems that commentators on this blog might also benefit from some thoughts on what constitutes useful data sources and plausible economics research, especially given the lags in peer review publication that seemingly give some people license to cite any ol’ thing on the web.
Here are my “guidelines” for my students:
- Don’t willy-nilly go researching on the web. With low costs of dissemination, anybody can write anything online.
- Peer reviewed publications in journals on average have had at least one level of screening; they are not “correct”, or necessarily “unbiased”, but at least they have had the benefit of some critique.
- If there are no published/forthcoming works, due to the rapidity of developments, then working papers or papers under submission (like a preprint) are a source. Not all working papers are created equally, though.
- Organized working paper series often have an internal review process, such as those from central bank and international organization research departments.
- Research associations (NBER, CEPR) don’t typically review the working papers, but there is an implicit screening in that only members can circulate papers. (By the way, this highlights the distinction between “publish” versus “circulate” – publishing in a peer reviewed journal requires – well – peer review).
- Relying on limited-membership publication could restrict one to “group-think” bias; for the association series I mentioned, I don’t think that that is a big worry – unless you think “group-think” encompasses the economic mainstream (i.e., if you’re looking for Marxian, Sraffian, or Austrian analysis, I admit they’re unlikely to show up).
- None of the foregoing says ignore an unaffiliated working paper. If one is unable to independently critique, it’s useful to know if the paper has been subjected to comment at conferences/seminars. There’s some in-group bias there since invitations depend on networks.
- BUT, the development of the internet means one can access many more critical analyses than in earlier times. For those with access to university libraries, the successor to the Social Sciences Citation Index, the Web of Science, is key. For those without, Google Scholar is a good substitute.
None of the foregoing guarantees one identifies the “right” research. But it reduces the likelihood of getting the random piece of research that just happens to validate one’s prior views – you know, like 22% is the relevant herd immunity threshold – or (more durable), income tax cuts typically pay for themselves.
Meta-analyses – quantitative analyses of estimates – can also signal when you’re taking a particularly iconoclastic view. If for example you’re citing an elasticity that is the most extreme out of 300 estimates, well, you might have reason to wonder. (See example re: minimum wage here.)
By the way, unbiased (once again, not necessarily correct) economic analysis and projections for the US are available at CBO, and unbiased recounting of developments and anaysis is now available at the (Library of Congress’s) Congressional Research Service — the Congress’s think tank.
https://www.cnn.com/2020/09/25/us/florida-ron-desantis-restaurants-open-trnd/index.html
Florida governor signs order clearing restaurants and bars to fully open
And they are seeing 100 coronavirus deaths per day. NY is having less than 4 deaths per day and our restaurants are still doing only outdoor seating with social distancing.
I guess this is why Senator Paul thinks Cuomo is doing it all wrong. Florida is pursuing the herd immunity approach which CoRev thinks is supported by all sorts of scientific research.
We now have experimental controls in place. Thanks Governor DeSantis! Your citizens are now guinea pigs so that the rest of us can quantify the value of what we are doing differently. What a guy! The dead people may not agree, but i am sure he is only thinking of science.
“For those with access to university libraries, the successor to the Social Sciences Citation Index, the Web of Science, is key. For those without, Google Scholar is a good substitute.”
Good places to do research if one is interested in actual research. But remember, CoRev and Bruce Hall have been ordered by their political masters to cite whatever Scott Atkins and Kelly Anne Conway recommend.
Useful. Great post.
“any ol’ thing on the web”.
Sort of like all those links Bruce Hall gave you proving that hydroxychloroquine was a safe and effective treatment for COVID19. How did that work out?
Personally, I was kind of upset Menzie didn’t mention Quora. All of the great “mathematical economists” around Shenandoah Valley quote it like the Bible. The archived pages of Quora are soon to be a permanent part of the Vatican Library. Or so Barkley Junior has informed me. Along with papers Barkley has assembled on the spread of dangerous viruses from science labs located near Wuhan wet markets. It’s part of the “deep state” conspiracy against donald trump, run by Dr. Fauci and some Chinese lab scientists. Barkley Junior will release the grand jury reports very soon.
What a waste of time, but BS Jr. is wasting his own time again.
SSCI, Web of Science, and Google Scholar are not outlets where articles or internet pieces are published. They are indexes. Whatever its quality, Quora is not an index, it is an outlet, one of varying quality. A completely inappropriate and irrelevant comment, BS Jr.
As for the origin of SARS-Cov-2, that remains unknown, including how it got to the notorious Wuhan “wet market.” Until we get actual new information on this, there is no point in further discussing this matter here as it is a mystery. I suggest you not waste peoples; time and attention any further on this thread with this sort of nonsensical drivel, BS Jr.
Not going to say it…….. Just hope ltr is reading today. PLEASE go get the shot ltr. You were born for this purpose ltr:
https://www.nytimes.com/2020/09/26/business/china-coronavirus-vaccine.html?action=click&module=Top%20Stories&pgtype=Homepage
Menzie, you still don’t fully understand HIT; “validate one’s prior views – you know, like 22% is the relevant herd immunity threshold“. Relevant herd immunity threshold is a set of calculations for a specific set of assumptions? Recent studies are comparing the results for homogeneous versus heterogeneous population assumptions.
Some of those assumptions are social policy related, masks, social distancing, washing hands, cleaning surfaces/areas, etc. Some of those assumptions are medical related to individual susceptibility, and demographic susceptibility, etc.
When variables associated with assumptions are changed, the end result of the HIT calculation also should. Do you actually believe that studies using different assumptions can not result in low HIT results?
CoRev: Well, sure, if people didn’t interact at all at distances less that 20 feet, lived alone, physically didn’t travel, HIT could be zero. I think I comprehend.
Menzie, as I thought you don’t understand. HIT indicates R=1. What you described or calculated was R=0 or suppression/Herd Immunity.
“The herd immunity threshold is the proportion of a population that need to be immune in order for an infectious disease to become stable in that community. If this is reached—for example due to immunisation—then each case leads to a single new case and the infection will become stable within the population, that is, R=1. If the threshold is surpassed, then R<1 and the disease will die out."
https://medical-dictionary.thefreedictionary.com/herd+immunity
Using your example I interpret your response you actually believe that studies using different assumptions can result in low HIT results. Now that we've established that simple proposition, is Rand Paul with an actual study reference more correct than Fauci with none? Moreover, has NYC UNDER ITS CURRENT IMPLEMENTED POLICIES REACHED THE HIT?
Do you understand your above conditions are a subset of the social Persistent Heterogeneities described in the PNAS study https://arxiv.org/pdf/2008.08142.pdf referenced so many times now?
CoRev Oh my. You’ve really stepped in it. Where to begin. First, if you can be bothered to read the very paper that you linked to, it says quite specifically that the effective rate of transmission is not linear. The definition you linked to is the definition used for an Rt calculated in the traditional homogeneous HIT SIR/SEIR compartmental model. Go read the paper. Specifically, look at where the authors put what they call the “lambda” value. It’s at the bottom of page 2. It is not linear transmission rate. You’ve completely botched the math. Second, even in the homogeneous HIT model the effective rate of transmission decreases. It is true that each case leads to one additional case when Rt = 1.0, but the “S” curve in the SIR model decays in a nonlinear way. I know you’re not going to understand the math, so let me provide you with a real world example. You’ll have to load “R” on your computer, but I’ll provide the code, so all you will have to do is copy and paste the commands. This real world case is based on an infectious outbreak at a girls boarding school in England. In the real case the initially infected population was 14, but I’ll simplify things to set it at one. This doesn’t change the result, but it will be easier for you to follow. The infectivity factor was 1.60 per student (meaning each girl came in contact with 1.60 other girls while infected. The mean infectious time was about 2.27 days. There were 763 girls in the school. So dividing 1.60 by 763 girls gives us a beta factor of 0.0021. A mean infectious time of 2.27 gives us a “nu” factor of 1 / 2.27 = 0.441. The HIT is reached when the remaining number of susceptible students equals 0.441 / 0.0021 = 210. This means that the number of new infections will start to fall when there are 210 girls who have not yet been infected. That is one way to express the HIT. Or, if you’d rather see it expressed in a more familiar way:
R-naught = 1 – 1/(1.60 / 0.441) = 1 – (1 / 3.628) = 1 – 0.2756 = 0.7244, or an HIT of 72.44%, or 72.44% * 763 girls ~ 553 current infections + recoveries.
Now, here’s the “R” code:
library(deSolve)
pars <- c("beta"=.0021, "nu"=.441)
times <- seq(0,20,.01)
y0 <- c(762,1,0)
sir <- function(t,y,p) {
yd1 <- -p["beta"] * y[1]*y[2]
+ yd2 <- p["beta"] * y[1]*y[2] – p["nu"]*y[2]
+ yd3 <- p["nu"]*y[2]
+ list(c(yd1,yd2,yd3),c(N=sum(y)))
+ }
sir.out ” signs in front of each new command. Also note that a “+” sign simply continues a command line and you should not put a “>” in front of it. I wanted to removed the “>” signs because they can cause problems when posting.
You will see two outputs. There will be a graph that shows the infection (green) curve peaking at the same point where the blue (susceptible) and red (recovered) curves intersect. Then look at the output on the main screen. At time period 6.65 you will see that the number of girls remaining in the susceptible category is 210, just as the HIT calculation predicted. Then look at the number immediately to the right. That’ the number of girls currently infected (282.343485). And note that this is also where the peak is on the green line. The infection numbers immediately above and immediately below this value are all less than 282.343485. In other words, the HIT is 210 remaining susceptible girls, or the point at which there are 282 girls currently in an infectious state. You can express the HIT either way. You can also see that the total number of currently infected plus recovered equals 282.34 + 270.29 ~ 553, which is what was calculated above. That’s another way to view the HIT. All are equivalent. Also notice that all three of the curves are nonlinear, which shows that the linear Rt value decays in a nonlinear way. The instantaneous Rt is linear, but over time it decays in a nonlinear way.
I have a whole lot more to say about your complete misunderstanding of heterogeneous HIT, but this post is already way too long. You really need to retire to some old folks home and drool in front of the television.
Okay, I knew that it wouldn’t like some of the code. After this line:
+}
you will need to add this:
sir.out <- lsoda(y0,times,sir,pars)
sir.out
plot(sir.out[,1],sir.out[,2],type="l",col="blue",xlab="Time",
+ ylab="Compartment Size")
lines(sir.out[,1],sir.out[,3],col="green")
lines(sir.out[,1],sir.out[,4],col="red")
legend(8,95,c("S","I","R"),col=c("blue","green","red"),lty=c(1,1,1))
2slugs,
You are attempting to obfuscate again by posting your irrelevant R code.
Corev is saying that the effective R, RE = 1, when the disease begins to die out. That’s trivially true and correct.
Rick Stryker I agree that it’s trivially true, although I think you meant to say when the effective Rt is less than 1, not equal to 1. But I don’t believe CoRev understands what it means in the same way that you and I understand it.
Wow! 2slugs finally read the article and like most still misunderstands the point. Re is, has been universally accepted, as the meaningful calculation for “Herd Immunity” and the immunity inflection(s) point is called the HIT!!! It is from Re the effectiveness of policies can be established. Polcy effectiveness then can be applied to environments like the food service industry.
Even Wiki has an excellent description of the concepts: https://en.wikipedia.org/wiki/Basic_reproduction_number
“R_{0} is not a biological constant for a pathogen as it is also affected by other factors such as environmental conditions and the behaviour of the infected population….
The most important uses of R 0 {\displaystyle R_{0}} R_{0} are determining if an emerging infectious disease can spread in a population and determining what proportion of the population should be immunized through vaccination to eradicate a disease…
Heterogeneous populations
In populations that are not homogeneous, the definition of R 0 {\displaystyle R_{0}} R_{0} is more subtle. The definition must account for the fact that a typical infected individual may not be an average individual….
Effective reproduction number
In reality, varying proportions of the population are immune to any given disease at any given time. To account for this, the effective reproduction number R e {\displaystyle R_{e}} R_e is used, usually written as R t {\displaystyle R_{t}} R_{t}, or the average number of new infections caused by a single infected individual at time t in the partially susceptible population. It can be found by multiplying R 0 {\displaystyle R_{0}} R_{0} by the fraction S of the population that is susceptible. When the fraction of the population that is immune increases (i. e. the susceptible population S decreases) so much that Re drops below 1, “herd immunity” has been achieved and the number of cases occurring in the population will gradually decrease to zero.[37][38][39] ”
Care to explain why the current trend of cases are rising and the trend for deaths are falling? Or care to explain how Rand Paul is wrong about NYC and herd immunity? Who was more correct Paul or Fauci? It was that discussion that caused this long series of articles and their comments.
2slugs grudgingly admits: “Rick Stryker I agree that it’s trivially true, although I think you meant to say when the effective Rt is less than 1, not equal to 1.”
Grudgingly, because of his closing sentence. “But I don’t believe CoRev understands what it means in the same way that you and I understand it.”
From the beginning had 2slugs, Menzie, PGL, Baffled, and the many others commenting understood what the HIT meant, then they would have understood that Rand Paul and the others referenced (even me) were talking about Re ( the effective reproduction number Re is used, usually written as Rt, or the average number of new infections caused by a single infected individual at time t in the partially susceptible population.) From that ole Wiki reference. They instead focused on the Herd Immunity portion. Fauci may be excused as an epidemiologist, because he’s used to thinking in terms of R0 and vaccines. For the rest of you there is no excuse.
Why is it the liberal/Dems can not read and comprehend questions re: their established beliefs?
for the life of me, i still cannot figure out what the incoherent corev is arguing about. its like he is arguing in circles. there seems to be no coherent thought leaving his brain these days-not that there ever was before. look what reading his gibberish did to parscale.
Baffled, how shocked am I that one of the key actors who never understood the original issue still doesn’t. Here’s a hint. It was about Re and not R0, because “Immunity is a continuum.” Who said that? Why is that true?
@ baffling
The main “takeaway” (as the kiddies say) here, is that CoRev spent years owning (or was it managing??) a restaurant, but thinks social distancing for food workers to protect themselves from a highly contagious respiratory virus that kills people is “authoritarianism” because it disallows him from sitting at the stool seats near to the bar. That’s pretty much all you need to know, aside from the fact CoRev doesn’t know the difference between spot prices and futures for farm commodities.
Plus, CoRev is confused why he couldn’t find workers for his restaurant when he’s so friendly to labor.
@ baffling
One day we’re going to teach CoRev that MAGA policies that are killing export markets for American food producers lowers domestic prices on those same farm products. But that’s a tough mountain to climb when CoRev can’t look at a simple market price and tell you if it’s a spot price or a futures price. CoRev has to complete some baby steps across the living room before we can give him his first tricycle.
“Immunity is a continuum.”
i already explained this issue to you corev. you seem too stoopid to retain the knowledge, unfortunately. this is why you continue to mix up asymptomatic with immunity.
“Some of those assumptions are social policy related, masks, social distancing, washing hands, cleaning surfaces/areas, etc. ”
OK – if people follow the advice of Dr. Fauci, great. But people like you, Rand Paul, and Bruce Hall have been asserting people do the opposite. Good grief CoRev – you are all over the map.
sometimes it is important to note some examples explicitly. for example, it would be helpful to avoid publications that come directly from a “think tank” when writing your own research paper. if the think tank report ends up as a peer reviewed article, great. but many such reports are simply “circulated” by the think tank itself, with no review other than approval from those who provide the think tank its money. this is in line with what menzie has noted. but some folks need to have the warning read to them a little more explicitly. politicians like to reference think tank reports. but politicians are not writing a research paper. for many reasons, students should not mimic their behavior.
Insist that students gather/create data.
Bob Flood: Excellent, glad you said that. I’ve required my undergrads to use FRED – god’s gift to teachers – to download, look at data, instead of me pre-digesting it. Having them do it for papers (i.e., disallowing them from cutting and pasting other people’s graphs) is part of the mission.
After the tidal wave of junk science ala CoRev on herd immunity, our host asked him if he ever heard of Optimal Policy under Uncertainty. Macroeconomics often think of monetary policy under uncertainty which has a 50 year history of great literature starting with William Poole’s article:
https://www.jstor.org/stable/1883009?seq=1
Optimal Choice of Monetary Policy Instruments in a Simple Stochastic Macro Model
William Poole
The Quarterly Journal of Economics
Vol. 84, No. 2 (May, 1970)
And there then is this 2019 survey:
https://econpapers.repec.org/bookchap/elgeebook/16504.htm
Optimal Monetary Policy under Uncertainty, Second Edition
Richard T. Froyen and Alfred V. Guender
Abstract: This book provides a thorough survey of the model-based literature on optimal monetary in a stochastic setting. The survey begins with the literature of the 1970s which focused on the information problem in policy design and extends to the New Keynesian approach of the 1990s which centered on evaluating alternative targeting strategies. New to the second edition is consideration of research since the world financial crisis on the role of financial markets and institutions in the conduct of monetary policy.
But the discussion is addressing COVID-19. Hey an idea for CoRev. Maybe he can join Senator Paul and write a paper entitled Optimal Policy Under Wishful Thinking.
Menzie,
There you go again, misleading the students with your misleading “funnel plot” metastudy. I don’t feel like re-litigating this, so I’ll just quote David Freedman : “Finally, with respect to meta-analysis, our recommendation is simple: Just say no. The suggested alternative is equally simple: Read the papers, think about them, and summarize them. Try our alternative. Trust us: you will like it.”
Sorry, Stryker, but this is a stupid and irrelevant comment. You misrepresent what Menzie posted. He presents 8 main points, all of which are very reasonable and none of which this dumb comment has anything to say about. Importantly he does not rule out any source, although he gives good advice on why some papers one might be more willing to trust than others, such as one being subjected to some sort of review while another is not. But he makes it clear that an “unaffiliated” paper may still be worth not ignoring, He never says “do not read a paper,” which seems to be the implication of you dragging in this obvious quote from Freedman, which is simply irrelevant.
Oh yes, the quote was about relying on meta-studies. But look carefully at what Menzie wrote about how to use those, which was in an additional piece of advice after his main 8 points. He noted that they may be useful for showing whether or not a particular study might be an outlier from others or not. That is completely correct and is not at all obviated by your quote from Freedman. What Freedman says is “read the papers,” but nowhere does Menzie say “don’t read the papers.” He simply and accurately notes that a meta-study can give one a heads-up that a particular paper may be disagreeing strongly with most others, which can be useful to know when one reads it. He never said, “do not read the outlier paper.” The obvious point is that one should read such a paper with greater care and possible skepticism, especially if it is lacking the sort of backing described in his 8 points, such as ever having been reviewed by anybody. Even if it has not, he does not say “do not read it.” He effectively says “read it with care and caution,” which does not at all go against what Freedman says.
Just what was the point of you putting this stupid piece here? Are you out to remind us of how you so often say such silly things here? Please do not try to pose as somebody more expert on this sort of thing than Menzie is. You have attempted this before and only ended up flat on your face. We know better, so just stop wasting peoples’ time with your blubbering baseless egomania.
“CoRev
September 27, 2020 at 1:09 pm
Menzie, as I thought you don’t understand. HIT indicates R=1. What you described or calculated was R=0 or suppression/Herd Immunity.”
This has become the dumbest thread ever. CoRev keeps making things up as he goes. Moving the goal posts is one thing but taking them to another country is another. Rand Paul was not talking about R=1 at all. He was claiming that most people in NY were already immune to this virus, which clearly is not true.
OK then Paul tried to change his tune sort of like CoRev keeps doing. Oh yea – we may or may not be at R < ! here but precisely because we were social distancing and wearing masks. But note in mid Brooklyn they let up and cases are rising. So where R stands is not entirely clear.
What is entirely clear is that Rand Paul wants all states to follow Florida. Well guess what – they are seeing over 100 deaths a day even on a 7-year average.
But of course CoRev will keep making shit up as he goes. That is what he does.
rand paul argues his position because he already was infected by the virus, and acted very irresponsibly while still in an infectious stage. he is projecting his immunity onto the behavior of others who do not possess the same immunity. as a medical professional, it is unconscionable to do what he is doing. his father has been wrong on just about everything for the past four decades. the apple does not fall far from the tree.
Up pops the world worst analyst confirming Rand Paul: “Oh yea – we may or may not be at R < ! here but precisely because we were social distancing and wearing masks." BTW, that's Re<1 or if you prefer Rt<1. The two are both used interchangeably. Unless you are reading the Chinese literature who do SOMETIMES use R instead of Rt/Re. Get the terms correct, please. Without that you are showing your ignorance.
Yano, snark is not an asset when FLAUNTING your ignorance. Just saying.
2slugs,
I do mean RE = 1. That’s the point at which the number of infections is maximized. Immediately after that point, the number of infections begins to drop, at least in the simplest SIR model.
Corev’s comment was completely sensible. Why assume that he needs some sort of lesson in Covid modeling. He didn’t claim that RE was “linear” as you alleged. He just said that R0 and RE are not constants of nature, since they vary with behavior, biology, policy, etc. That’s true.
You don’t need to solve the ODEs numerically in R. If you want to calculate the maximum number of girls infected, then you could do that analytically. But the maximum number of girls infected is irrelevant to the point that Corev was making, so why bother?
“I do mean RE = 1. That’s the point at which the number of infections is maximized. Immediately after that point, the number of infections begins to drop, at least in the simplest SIR model.”
an R value of 1 with a significant number of people in the community still infectious and susceptible means there will still be a lot of future infections until the virus disappears. considering we do not know R with great accuracy or precision, conditions that remain near to 1 should not be considered a victory. they can easily relapse into defeat.
rick and corev are implying as long as you get Re to one or below, the work is done. 2slugs shows that is not really the case. yes the virus will eventually die out, but it can still do ALOT of damage on the way out the door with an Re around 1. and since we know the R values are a function of how people behave (social distancing, mask wearing, etc), the longer you keep the R value around 1, the greater chance something happens to force it back above 1. then you get exponential spread in your community again. this can be avoided if we choose to do so.