I get the impression that people mix up things that are logically distinct, even if they might overlap. For instance, one reader says criticizes the failure to use big data at the Fed [1] [2] (see this post for a rejoinder) and in a closely adjoining comment, contrasts the GDPNow series run by the Atlanta Fed and the absence of a global counterpart at the Board. Let’s define some terms, so people who are inclined toward confusion become less confused.
Nowcasting: Reichlin et al. (2010) “… define nowcasting as the prediction of the present, the very near future and the very recent past. Crucial in this process is to use timely monthly information in order to nowcast key economic variables, such as e.g. GDP, that are typically collected at low frequency and published with long delays.” For instance, the Atlanta Fed’s GDPNow uses monthly data to nowcast GDP, which comes in at a quarterly frequency. If you look at the input series, you’ll notice that they are pretty much from conventional releases from the US government, Fed, or surveys like those from the ISM. I would not characterize GDPNow as a use of big data (although some might disagree). For an application, see here.
If you’re going to purport to know something about nowcasting, you might want to know something about MIDAS (application in an Econbrowser post here).
Big Data: “Big Data refers to data sets of much larger size, higher frequency, and often more personalized information.” This definition is going to encompass an enormous set of types of analyses, so any specific list will be arbitrary. However, one can get a feeling for what counts, by referring to instances cited on Econbrowser: OECD Weekly Tracker (using GoogleTrends), using GoogleTrends, pandemic impacts (using Google mobility trends), Delta variant (Google, Apple), and for Asia over the pandemic. For the 15 months of the pandemic, the Dallas Fed maintained a “mobility and engagement index”, used in this post. Finally, an application of big data to China was in a guest post by Laurent Ferrara (via QuantCube); this was a nowcast of quarterly consumption, so was an application of both nowcasting, and big data.
Enormous data sets and/or lots of variables, you’ll need something to figure out which data to use and which not to use. Lasso is one of those things you need to understand if you’re going to go down this route. (To me the latest version of neural nets, but I’m sure there are subtleties I’m eliding here.)
If one thinks these are things not being considered for use at central banks, investment bank research departments, and international organizations, even if they are not being reported for public consumption, I think that person is confused.
(As for whether the Fed has considered using things like satellite data for forecasting / nowcasting as Mr. Kopits queried, see e.g., Hakkio and Nie (2018), Nie and Oksol (2018), Clark et al. (2020). Whether they maintain a series that’s updated over time, I can’t say (and they won’t circulate until it’s “ready for prime time”, even if they were to use it internally). I mean, let’s assume Mr. Kopits uses spreadsheets; is he willing to publicize all of them?
We can make this very simple by noting there is a wide array of topics where SK has no clue what he is talking about but that has never deterred him from ranting on and on and on and on as if he is the only person who gets it.
I really am kind of amazed he never made official membership into the orange abomination’s MAGA staff. Would anyone here believe me if I said I’m not even joking with that prior sentence?? I would not be angry if you thought I was lying, but I’m not joking. It genuinely kind of borderline shocks me Kopits never got official membership into MAGAworld.
What is the purpose of nowcasting?
Why is GDP important?
Can you improve welfare much more efficiently using an inflation-adjusted basic income?
Why does the Fed rely on surveys that have large ststistical sampling error for employment, wages, etc.? Is it because it’s none of the Fed’s business, what a private firm pays or makes?
but, rsm, you have already answered the question of whether or not one can “improve wlefare” by “using an inflation-adjusted basic income” when you assured us that “inflstion is just irrational noise.” If that is so, why are you wasting our time with such a question?
One thing I have always prided myself on, is not talking about things I don’t know about (that’s not to say I don’t “step in it” on rare occasion). I only consider myself slightly above average intelligence. That is to say, if I randomly ran into 10 people off of the street or at a public park, I’d be smarter that 6–7 of the 10, and roughly 3 of the 10 would be sharper than me. I just have always found, even when you’re not “the sharpest tool in the toolbox” or “brightest lightbulb” or pick your nonsense metaphor of liking, people just respect you more when you keep your mouth shut in those moments that “you know that/what you don’t know”. Which admittedly happens much more than my ego can stomach, but you know, “there it is”. At least I’m going to let them guess IRL whether I know it or not, not prove to them I don’t by opening my mouth.
The call for more use of “big data” by someone who can’t even be bothered to use little dabs of data is telling. This blog is inundated with comments expressing strong opinions based on poor understanding of basic economics, with assertions about facts which misrepresent the facts, with claims about data from those who haven’t bothered to look at data. Kopits is pretty reliable at doing each of these. And here he is, demanding that others make more use of data. And oooo, let’s make sure the extra data are big, ’cause big is good.
Ego, dogma, innumeracy, economic illiteracy, hackery – all are bigger problems than a shortage of data or smallness of data or innacurate data or error bars. If people who want a voice in discussion of policy, and in policy itself, bothered to take the discussion seriously, the whole enterprise of policy making in a democracy would work a lot better. Instead, we have preening, posing, self-aggrandizing nobodies getting in the way of real discussion.
Just look a Kopits’ “show me, show me, show me” harangue. Lazy ignorance and ego. Menzie was nice enough to comply, showing Kopits to have been utterly misinformed, as usual.
If you knew anything about this, you’d realize that the Fed agrees with SK that developing a big data function would be useful to the Fed. A number of the regional banks have spent a lot of resources hiring computer scientists and data science specialists, particularly the NY Fed and the San Francisco Fed, which is developing a team that will serve big data needs system-wide. Boston and Chicago and others are also developing these capabilities. These Fed banks are hiring people with backgrounds in Apache Spark and Hadoop, AWS, datalakes, Java, Scala, python, c++, etc.etc.etc. i.e., big data specialists. I don’t know whether the Fed’s investment in big data amounts to SK’s suggested $50-100 million per year, but I do know the investments the Fed is making are serious and substantial. Time will tell whether these investments will bear fruit.
There is no reason that SK would have been aware of the Fed’s activities, not being a Fed insider or knowing people in the system, but he was certainly on the right track in thinking the Fed should be developing big data capabilities. Menzie is also unaware of what the Fed has been up to, since he responded with irrelevant points about fed academic research, big data conferences, and Fed governor speeches.
Wait, there’s a small clean area on the wall where the scat you were throwing didn’t stick to. Try a third time and see who takes the bait.
If Kopits is unaware of the Fed’s interest in big data, then he is exactly as I have portrayed him, blathering in ignorance. Big consultant man poser.
But it’s nice of you to come to his defense, one poser helping another.
Isn’t cute when the Usual Suspects write literally thousands of pages of utter BS and then have one of their colleagues find one single sentence in those long winded rants that they can defend. They must be paid by the word.
Menzie,
Your response in this post and the last shows that you know very little about machine learning and big data. The “reader” said that he wished the Fed had set aside a budget of $50-100 million per year to develop a big data technology function that would develop data sets that would be used by the Fed internally for decision making and would be released to the public for their own research purposes. Since you don’t understand that big data is about computer science, you responded with the irrelevant points that some Fed economists are using large data sets in their academic research (just as academic economists do), that the Fed has organized some big data academic conferences (just as Universities do), and that some Fed officials have made some speeches about big data. That’s all true but beside the point. The reader was suggesting that the Fed should be hiring computer scientists to set up a big data function. You didn’t answer is point.
If you had understood this situation better, you would have pointed out to the reader that many regional Fed banks are already spending substantial resources to establish just such a big data function. For example, the NY Fed is way into this and Chicago and Boston are also developing these capabilities. These banks are hiring computer scientists, not academic economists. And they are looking for expertise in distributed databases, python, system programming in c++, etc, etc. to develop the data sets and analysis the reader said he wanted to see. You could have pointed this out to the reader without calling him a “dolt” and “buffoon” as you so rudely did.
You further show your ignorance when you quote a definition of big data written by economists. Big data is a computer science concept. It refers to large data sets that are usually unstructured and that are therefore hosted in a distributed server system rather than in a conventional database. The expertise you need for big data is facility with applications like Apache Hadoop, Spark, etc or more specialized systems such as Google Earth Engine.
Just to correct some of your misconceptions, Lasso is not a big data methodology. It is a machine learning technique that is used to improve the forecasting ability of simple and more advanced regression and statistical models. It can be used on small as well as big datasets. Similarly, neural networks are not a big data methodology per se. They are a general methodology that can be used on small as well as big datasets when you need to approximate a complex non-linear function or relation. Neural networks can also be very useful in reinforcement learning, which isn’t about big data.
Rick,
I hope you understand that neural nets are basically systems of layered nonlinear regressions.
rick, your responses to prof chinn indicate you really did not read and absorb what he wrote. he was not inaccurate in his response. seems you simply wanted to take an opportunity to brag about how much you know about big data. kind of like the lessons you used to provide about your wonderful r0 models. let’s take a few minutes and look up something on wikipedia, then share your extensive “knowledge” with others.
Professor Chinn,
The MIDAS link provided above discusses combining a VAR with MIDAS. I understand what a VAR is, and I understand how to compute a MIDAS regression in EViews, but do not understand what the authors say about combining a MIDAS model with a VAR at least using EViews. A simple example would provide much needed enlightenment for this reader.
@ AS
This looks very complex to me, but I’m hopeful it might help you somehow.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2593796
There’s a download link after the link jump. God bless.
Moses and Professor Chinn,
Moses, thanks. I would need to be in a class to follow the paper.
The prior version of EViews did not offer a MIDAS VAR.
Embarrassingly, I now notice that EViews 11 offers a MIDAS VAR.
I owe an apology to Professor Chinn. It looks like EViews has a YouTube example for a MIDAS VAR or mixed frequency VAR.
Professor Chinn, please accept my apology. I will watch the YouTube example.