Economics

Introduction

The notebook titled Economics focuses on two of the most recent economic issues of the world : the Sub-Saharan African Growth and the economic condition of Latin America and Paraguay. Datasets for both of these topics for the notebook were provided

Economics Notebook

by the International Monetary Fund. The IMF website provides a large and formidable list of  data indicators to choose from and extracting the relevant datasets proved to be a tough task. The method adopted for choosing the datasets has been described below. Analysis and visualization for the notebook was done using Apache Spark, the default visualization capabilities of Zeppelin( as described in the introductory post), the D3 libraries using html display and Helium Application. As in the previous notebook, the datasets were downloaded first using the shell interpreter and then loaded and converted into appropriate 'DataFrames' for analysis by using a custom 'explode()' function that takes the input dataframe and returns a dataframe that can easily be queried and and registered as tables. The first two paragraphs in each section perform these data fetching and loading tasks while the rest are used for analysis.

 Getting the Data Sets

Once the topics for analysis were chosen, the datasets relevant to them had to be figured out. For this purpose, certain blogs and publications such as those on the IMF website itself, were referred and the list of data indicators was manually scanned to extract the needed datasets. For example, the IMF site has two good articles on the recent advances in economic growth by Africa and Sub-Saharan Africa, links to which have been provided in the notebook. With the aid of these posts and publications, cause and effect diagrams were drawn to roughly find the interdependence of the indicators and their impact on the economy. For example, in the case of Africa the recent slowdown can be examined by first considering the variation in growth by country, revenues, commodity prices and government policies. Similarly for other topics and indicators.

Analysis using Apache Spark

1. The African Growth Story

 

Sub-Saharan Africa had been experiencing good economic growth for the last decade. Only recently has the growth slowed down. In the first topic of the notebook, we examine the factors that have effects on the African economy and what the region might do to improve. Six representative nations have been chosen for the analysis : Nigeria, Central African Republic, Democratic Republic of Congo, Madagascar, Mauritius and South Africa.

African Growth Story

We begin with the Broad Money and Revenues for the region both of which have stayed fairly constant over time or have fallen over time, especially the revenues for Nigeria. Even the Current Account Holdings indicate that they will be low for the years to come as indicated by the clustering results. On the other hand, Nigeria has been able to get maximum national savings unlike South Africa and Mauritius which have low savings and high expenditures. The expenditures indicate that South Africa and Mauritius might have the highest debt but the pie charts reveal that initially Central African Republic and Congo had the highest debts even with low expenditures, although expenditures caused the debts for South Africa and Mauritius to increase in the subsequent years. The capital formation values were again high for Mauritius, Madagascar and South Africa.

Among the factors that have an impact on the economy, the Consumer Price Index has been considered first and it is observed that it has only been rising in the recent years. Increasing consumer prices are an indication of inflation, even the gross debt increases linearly with the consumer prices. The next factor, access to financial services has been measured by considering the total number of bank branches and the bank usage to indicate the level of financial activity in the region. From the charts drawn, it is revealed that only South Africa has a great number of Bank Branches and also above mean broad money. Some nations have been consistently performing well even in terms of the value of exports namely, South Africa and Nigeria. However, an important fact is revealed in the graph between the Current Account values and the value of exports : the current account values don't increase with the increase in the value of exports, this may attributed to the net imports being either equal to or greater than the value of exports for the region. Finally, we get to see that the total expenditures of the nations and the region are almost entirely dependent on the government expenditures, so to keep the expenditures and debt under control, the government policies are going to play a crucial role, this is confirmed even in the IMF report which states that a change of government and fiscal policies is all that is needed to put the region back on the track of progress.

2.  Latin America and Paraguay

 

An IMF publication recently reported that growth in Paraguay would stay resilient at 3% even amid regional slowdown. In the second topic for the notebook, we examine the economic condition of the region and analyze some of the factors that have been affecting it.

Latin America and Paraguay Economic Growth

 

We begin with the Balance of Payments and the Gross Debt. For the Balance of Payments, we find that for some nations such as Bolivia, it has fallen drastically over the recent years, while for others it has either been constant or is on the rise as in the case of Brazil and Paraguay. In terms of spending and the gross debt, Brazil takes the lead and Paraguay stands good with debt projections below 30% of the GDP. This is an important point, as illustrated in the graphs that follow, that have the GDP values on the left axis and the debt values on the right axis. As Brazil would be having the maximum GDP, it would also have the highest debt at almost 90% of the GDP. On the other hand, Paraguay which has low debt percentage and still increasing GDP would stay strong economically.

Next, we proceed to examine the factors that lead to the present economic condition of the region, beginning with the general government net lending/borrowing. In the correlation that we perform between the net lending/borrowing and the Balance of Payments, we get a correlation value of 0.6012, as is also indicated by the graphs. This shows that the lending and borrowing largely affect the Balance of Payments. The next factor to be examined is the net cash inflow from financing activities which has been consistently high for Brazil and has been swinging up and down for other nations. Only for Paraguay has it been rising over the recent years and chances are that it will come close to Brazil in the years to come. In the correlation that is performed between the cash surplus/deficit values as dependent variable and the cash expenditures and cash inflow values, we get negative weights for both the features. This can be explained by the larger negative weight of the expenditures which overshadows the positive effect of the cash inflows resulting in an apparent negative weight for the cash inflows. Two of the nations, Brazil and Argentina distinctly stand out in terms of the Purchasing Power Parity of the nations leading the other nations by a huge margin. Differences in the purchasing parities also explain the net change in stock of cash - nations having higher purchasing parity have less inflow or outflow of cash as compared to nations having lesser parities assuming debt liabilities remaining constant and the same is observed in the net cash stock graph plotted alongside. Finally, we take a look at the changing Consumer Prices and get to see that they have been fluctuating the maximum for Argentina reaching the maximum and the minimum in consecutive years. For others, such as Brazil and Paraguay, they have largely remained constant over time indicating greater stability of internal markets. In the pie charts that follow, we get to see how the most important sectors of internal markets share the consumer price indexes amongst themselves. Food, housing and miscellaneous items form the greatest share with the housing share slightly increasing over time.

Visualizations

The visualizations for this notebook were created using the default visualization capabilities of Zeppelin along with the custom graphs and charts using the D3 visualization libraries. Default visualizations are simple, only the dataframes containing the data must be registered as Tables and then they can be visualized using sql. As for the custom visualizations, they can either be written( D3 code ) inside html display system of Zeppelin which requires the data in good formed json format or the helium pluggable application can be used. The html display would only require the data to be visualized which can be provided using string interpolation. Running the paragraph would create the visualization. Its positioning and colors can be adjusted using css. 

Helium application can also be used to create visualizations by first getting the data generated by the spark repl interpreter. The data can either be repl result or in table format.

Helium Visualization

A simple helium application would simply extend Application class provided in the helium package of Zeppelin. Then to get the result of the previous paragraph, we first need to get the Interpreter Context corresponding to the currently running instance of Zeppelin which we can get using a static getter method of the 'InterpreterContext' class. The interpreter context can provide the resource pool from which we can get the result of the previous paragraph as a 'Resource' type, which can then be used to provide the data to the html or javascript file used to create the visualizations. Since the visualizations( the html and javascript ) are displayed through the angular display system, we first need to add the data to the 'angular object registry' to be able to use it in the html file. See sample demo here.