Month: November 2023

In today’s lecture,

1. we explore the mechanics of the housing market, paying close attention to how median home values have changed over time and how they relate to larger trends in the economy.
2. Our study shows the market’s varying highs and lows, just like a roadmap might.
3. While price dips or plateaus may indicate a cooling market owing to economic developments or shifting buyer attitudes, rising prices frequently imply a solid economy and significant housing demand, signifying buyer confidence.
4. These patterns are linked to more general economic factors like employment and interest rates.
5. For example, a robust job market might raise the ability to purchase a property, which in turn drives up prices, while changes in borrowing rates can affect the fervor of potential buyers.
6. Additionally, we observed possible seasonal patterns in the market.
7. Time series analysis was performed on ‘economic indicators’ data in order to understand underlying patterns such as trends, seasonality, and cyclic behaviour.
8. The data was arranged chronologically by time variable (columns ‘Year’ and ‘Month’).
9. I later combined the ‘Year’ and ‘Month’ columns into a single date time column to indicate time.
10. The time series data was also visualised using time series plots to observe patterns, trends, and seasonality in each economic indicator over time.
11. The time series was then decomposed into its components (trend, seasonality, residual) for analysis using techniques such as seasonal decomposition.
12. Seasonal decomposition is a time series analysis method that separates a time series dataset into three parts: trend, seasonality, and residual or error components.
13. To predict future values of economic indicators, a forecasting model such as ARIMA (AutoRegressive Integrated Moving Average) was used.
14. The data was then divided into training and test sets in order to train the model and evaluate its performance.
15. To assess the accuracy of the forecasting model, a metric such as Mean Absolute Error (MAE) was used, and the predicted values were compared to the actual values in the test dataset to evaluate the model’s performance.

1. In a VAR model, each variable is modeled as a linear function of past lags of itself and past lags of other variables in the system.
2. VAR models differ from univariate autoregressive models because they allow feedback to occur between the variables in the model.
3. An estimated VAR model can be used for forecasting, and the quality of the forecasts can be judged, in ways that are completely analogous to the methods used in univariate autoregressive modelling.
4. Using an autoregressive (AR) modeling approach, the vector autoregression (VAR) method examines the relationships between multiple time series variables at different time steps.
5. The VAR model’s parameter specification involves providing the order of the AR(p) model, which represents the number of lagged values included in the analysis.

1. After analysing the “food establishment inspections” dataset, it was determined that the decision tree technique would work best because it can be used for both regression and classification, can handle both numerical and categorical data, implicitly perform feature selection, robust to outliers and missing values.
2. They use a structure akin to a tree to represent decisions and their possible outcomes.
3. The nodes in the tree stand for features, the branches for decision rules, and the leaves for the result or goal variable.
4. Decision trees determine the optimal feature to split the dataset at each node based on a variety of parameters, such as information gain, entropy, and Gini index.
5. The algorithm recursively splits the dataset based on the selected features until a stopping criterion is

MTH522_Project2_Report

1. After running the DBSCAN algorithm, I went on to create OPTICS, which is an extension of it.
2. preprocessed the data by transforming the object values into numerical ones and normalizing/standardizing the data.
3. Determined the distance between each location using the Haversine distance formula.
4. The OPTICS algorithm was subsequently implemented as follows: optics = OPTICS(min_samples=5, metric=’precomputed’)optics.fit (matrix of haversine)
   collected cluster information, including reachability distances and core samples, and identified the noise sites.
5. In order to comprehend the spatial distribution, the clusters were also visualized on the map.

1. Used the latitude and longitude coordinates and relevant attributes.
2. Calculated the Euclidean distance to measure dissimilarity between locations.
3. Then calculated a pairwise distance matrix to represent dissimilarity between all location pairs.
4. Performed agglomerative hierarchical clustering algorithm to the distance matrix
5. Visualized the dendrogram, which displayed the hierarchical structure of clusters.

Here is an overview of the main conclusions drawn from the data analysis:
1. The “flee” variable is divided into four categories: not fleeing, car, foot, and other.

2. Following the incident, about 4,000 people remained in their place.*

3. About 1300 people ran away, some on foot and some in cars.

4. Method of Death: Over 7000 individuals were shot, while the remaining 8000 were shot and subjected to tasers.

5. Gender Distribution: Compared to females, men were more likely to be involved in the offenses.

6.Racial Distribution: A variety of racial backgrounds were represented, with whites and blacks making up the majority.

7.Geographic Distribution: The reported crime rate was greater in three states.

8.Indices of Mental Illness: Approximately 2000 individuals exhibited indicators of mental illness.

4.Danger Level: About 4000 individuals were recorded

1. K-Means and DBSCAN are two distinct clustering algorithms.
2. K-Means is a partition-based clustering method where you need to specify the number of clusters (K) beforehand, and it assigns data points to the nearest cluster centroid.
3. However, it can be sensitive to the initial placement of centroids and primarily performs hard clustering, meaning each point belongs to one cluster. K-Means also assumes that clusters are spherical in shape.
4. On the other hand, DBSCAN is a density-based clustering algorithm that automatically identifies clusters based on data density without requiring you to specify the number of clusters in advance.
5. It excels at identifying clusters of arbitrary shapes, handles noise or outliers, and can accommodate clusters of varying sizes.
6. While DBSCAN primarily performs hard clustering, there are extensions available to achieve soft clustering when necessary.
7. In summary, the choice between K-Means and DBSCAN depends on factors like the knowledge of the number of clusters and the data’s characteristics, as they have different strengths and use cases in the realm of data clustering.

1. I carried on working today and we talked about the population analysis using the specific police station data in class.
2.  I also went over the race analysis I’ve done and how I want to combine it with the specific police station data. Apart from that, I was trying to create a map using the locations, longitude, and latitude information so that I could examine the clusters and the regions where there were a lot of gunshots.
3. I was using the geopandas library for this, but it appears that the world map extension is no longer supported, which is why, when the individual point grouping is finished, there is an issue when trying to import the map.
4. I am looking for more libraries that I can utilize.