開課年級: 大四 研究所 (中文授課)
<br>學分數: 3
<br>課程目標: 
<br>1. be familiar with data analytic platform, software stack and tools for cloud computing and big data processing.
<br>2. do MapReduce coding for data processing problems.
<br>3. paper reading on the state-of-art data analytic platforms and their optimization techniques.
<br>
<br>先修科目: None
<br>
<br>課程大綱:
<br>1. Introduction of Cloud Computing & Big Data
<br>
<br>2. Apache Hadoop Data Analytic Software Stack
<br>2-1: File System: HDFS, GFS (Google File System)
<br>2-2: Parallel Processing: MapReduce, Hadoop
<br>2-3: NoSQL Database: Big Table, HBase, Hive
<br>2-4: Resource Management: Hadoop-YARN
<br>
<br>3. MapReduce Basic Implementation, Algorithms & Application
<br>3-1: Inverted Index
<br>3-2: Page Rank
<br>3-3: Graph Algorithm
<br>3-4: Hadoop Programming
<br>
<br>4. MapReduce Advanced Scheduling & Optimization
<br>4-1: Locality Aware Delay Scheduling
<br>4-2: Heterogeneous Environment Scheduling 
<br>4-3: Load balancing
<br>
<br>5. Query Processing Optimization
<br>5-1: Hadoop DB
<br>5-2: Query Optimization
<br>
<br>6. BDAS(Berkeley Data Analytic System) In-Memory Computing 
<br>6-1: Mesos
<br>6-2: SPARK
<br>6-3: SHARK
<br>
<br>
<br>
<br>指定用書: Data-Intensive Text Processing with MapReduce, Jimmy Lin and Chris Dyer. Morgan & Claypool Publishers, 2010.
<br>參考書籍: 
<br>A. 雲端程式設計 入門與應用實務
<br>B. Hadoop: The Definitive Guide, O'Reilly, 2009
<br>C. Paper reading:
<br> 1. The Google File System. SOSP-03, pages 29-43.
<br> 2. MapReduce: Simplified Data Processing on Large Clusters. OSDI 2004, pages 137-150.
<br> 3. MapReduce and Parallel DBMSs: Friends or Foes? Communications of the ACM, 53(1):64-71, 2010.
<br> 4. Bigtable: A Distributed Storage System for Structured Data. OSDI 2006, pages 205-218.
<br> 5. Apache Hadoop YARN: Yet Another Resource Negotiator. SOCC 2013.
<br> 6. Hive – A Petabyte Scale Data Warehouse Using Hadoop. ICDE 2010.
<br>
<br> 7. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. Euro-Sys 2010.
<br> 8. Scarlett: Coping with Skewed Content Popularity in MapReduce Clusters. Euro-Sys 2011.
<br> 9. Interference and Locality-Aware Task Scheduling for MapReduce Applications in Virtual Clusters. HPDC 2013
<br> 11. Balancing Reducer Skew in MapReduce Workloads using Progressive Sampling. SOCC 2012.
<br> 12. Coupling Task Progress for MapReduce Resource-Aware Scheduling. INFOCOM 2013.
<br>
<br> 13. HadoopDB: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads. VLDB 2009.
<br> 14. SciHadoop: array-based query processing in Hadoop. SC 2011.
<br> 15. Spark: Cluster Computing with Working Sets. HotCloud 2009.
<br> 16. Shark: SQL and Rich Analytics at Scale. SIGMOD 2013.
<br> 17. Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. NSDI 2011.
<br> 18. HaLoop: efficient iterative data processing on large clusters. VLDB 2010
<br> 19. The case for RAMClouds: scalable high-performance storage entirely in DRAM. SIGOPS 2010
<br>
<br>
<br>
<br>教學方式:
<br>   講義授課上機實驗
<br>
<br>教學進度:
<br>   一周一個講義或實驗
<br>
<br>成績考核
<br>   Lab: 30%
<br>	LAB1: Hadoop (HDFS & HBASE)
<br>	LAB2: MapReduce
<br>        LAB3: SPARK
<br>
<br>   Programming HW: 30%
<br>        HW1: Inverted Index
<br>        HW2: Page Rank
<br> 
<br>   Final Project: 25%
<br>        Build a Search Engine using the open source, algorithm learned from the class.
<br>
<br>   Course Participation & Quiz: 15%