⏳ VTech Time Series Prediction

Building and developing a time series deep learning model to predict the VTech (Vertical Total Electron Content) which are the total number of free electrons relesed by the ionisation of the ionosphere caused by solar activities such as solar storms, solar winds and solar flares. For this model we obtained data from 2 sources:

• OMNIWeb Data Explorer
• Dataset provided from the EgSA for the years 2022,2023 and half of 2024

📂 Dataset

We collected data from the 2 resources listed above. Each contained different parameters depending on the available readings.

• The dataset obtained from the OMNIWeb we specified the time span from Jan 2022 to May 2024 sampled every hour, the paramaters we needed were:

• The dataset obtaned from the EgSA had many parameters as it was obtained from the readings of the agencie's ground station, it is sampled at 2 reading per second:

🔄 Data Cleaning

1. The data obtained from the OMNIWeb, we found that there are many outliers in the data, and in order to remove the outliers we did the following:

• We took the average reading from the rows above and below the outling rows and replaced the outling numbers.

2. Due to the vast differences in sample rate between the 2 data resources we had to resample the datasets in order to be able to concatenate them in one combined dataset we decided to do the following:

• In the data given to us by the EgSA we took the average reading for each hour in order to be able to concatenate this data with the OMNIWeb data and create a dataset we can fit into our model.

🛠️ Technologies Used

DL.ipynb

• TensorFlow/Keras
  • tensorflow.keras.callbacks.EarlyStopping
  • tensorflow.keras.layers.LSTM, GRU, Dense, Dropout, Bidirectional, Input, Attention, Concatenate
  • tensorflow.keras.models.Model, Sequential
  • tensorflow.keras.optimizers.Adam
  • tensorflow.keras.regularizers
  • tensorflow.keras.callbacks.ModelCheckpoint
• Scikit-learn
  • sklearn.model_selection.train_test_split
  • sklearn.decomposition.PCA
  • sklearn.metrics.mean_squared_error, r2_score
  • sklearn.preprocessing.MinMaxScaler
  • sklearn.linear_model.LinearRegression
• SHAP for explainability (shap)
• Matplotlib for visualization (matplotlib.pyplot)
• NumPy for numerical operations (numpy)
• Pandas for data manipulation (pandas)

insights.ipynb

• Matplotlib for visualization (matplotlib.pyplot, matplotlib.colors)
• Seaborn for advanced data visualization (seaborn)
• NumPy for numerical operations (numpy)
• Pandas for data manipulation (pandas)

Preparing_data.ipynb

• Matplotlib for visualization (matplotlib.pyplot)
• NumPy for numerical operations (numpy)
• Pandas for data manipulation (pandas)
• TQDM for progress bar (tqdm)
• Datetime for handling date and time (datetime)
• OS for interacting with the operating system (os)

🤝🏼 Contributers

Name
Abdulrahman Hisham
Hamza Elghonemy
Jannat Allah Sabry
Youssef Husseiny