The National Bank of Canada or NBC is an integrated group that provides comprehensive financial services to individual, small and medium-sized business clients net of large corporations in its home market, as well as specialist services internationally.
NBC Financials Market has two projects:
- The first one is moving a on-premise Cloudera Hadoop 18 nodes Cluster to Amazon Web Services Cloud infrastructure.
- The second one is creating a Proof of Concept for an analytical models factory (machine learning black-box) that select best algorithm to predict stocks and market prices.
In this context I was asked to:
- Define the Cloudera Hadoop Cluster migration strategy and road-map to the AWS Cloud.
- Define and implement AWS data security policies for the cluster.
- Define and implement AWS / EMR / Athena cluster monitoring strategy.
- Review and support developers for Spark / MapReduce batch jobs within the migration period.
- Act as Big Data trusted advisor (architectures and technologies).
- Design a real-time analysis platform to ingest / forecast in real-time market data.
- Analyze Murex application logs in order to create a predictive maintenance model.
- Work on the POC of the analytical models factory and implement it on AWS using SageMaker.
- Contribute as a technology advisor to implement a fraud detection PoC on Google Cloud Platform.