OPTIMIZING ENSEMBLE FORECAST USING SPARK

PROJECT GOAL :
    Ensemble forecast model uses a combination of different forecasting algorithms to predict demand for a product and uses an ensemble of these values to come up with a prediction. The goal of this project is to use Data platform for signals and use Spark for executing Ensemble forecast models.


TASKS
                            TASK                               STATUS               COMMENTS

    DATA PIPELINE TASKS:

        Creating bigfoot fact table for sales,orders           done                 retail_ip__dfp_sales_signals_fact
        Creating bigfoot fact table for pageviews              done                 retail_ip__dfp_product_fsn_pvs_fact (Full Year data for pvs)
        Creating bigfoot fact table for offers,prices          Already present      retail_ip__dfp_offers_fact,retail_ip__dfp_orders_fact
        Fetch Sample data to use in forecasting algorithms     done
        Sample Spark Job                                       not done             to be done after all forecasting algos are implemented
        Missing Pageviews Treatment                            Working on it        missing page views ,replaced by average of fsn from other dates.
        Bulk Order Normaliziation                              To do

    FORECASTING ALGORITHMS:

        Arima                                                  done                 used spark-timeseries library
        Croston                                                done                 Implemented the algorithm in Scala.