Difference between revisions of "Team:NYMU-Taipei/HP/Gold-Forecasting Website"

(Created blank page)
 
 
(39 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{NYMU-sheet}}
 +
{{NYMU-Taipei}}
  
 +
<html>
 +
 +
<head>
 +
 +
<link href="https://fonts.googleapis.com/css?family=Product+Sans" rel="stylesheet" type="text/css">
 +
<meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;">
 +
<link href="https://2016.igem.org/Team:NYMU-Taipei/StyleSheets/sheet?action=raw&ctype=text/css" rel="stylesheet">
 +
 +
<script>
 +
for (var i = 0; i < images.length; i++) { var image = images[i], width = String(image.currentStyle.width); if (width.indexOf('%') == -1) { continue; } image.origWidth = image.offsetWidth; image.origHeight = image.offsetHeight; imgCache.push(image); c.ieAlpha(image); image.style.width = width; }
 +
</script>
 +
 +
<style>
 +
</style>
 +
 +
</head>
 +
 +
<body>
 +
 +
 +
    <div class="prototypeprototypesp">
 +
        <div class="imageimage00">
 +
            <img src="https://static.igem.org/mediawiki/2016/2/23/T--NYMU-Taipei--%E5%88%86%E9%A0%81_human_practice_%E6%89%8B%E6%A9%9F%E7%89%88_gold.jpg" width="100%" height="100%" />
 +
        </div>
 +
    </div>
 +
 +
    <div class="prototypeprototype">
 +
        <div class="imageimage00">
 +
            <img src="https://static.igem.org/mediawiki/2016/9/91/%E5%88%86%E9%A0%81_human_practice_%E5%8A%A0%E5%AD%97%E7%89%88_gold.jpg" width="100%" height="100%" />
 +
        </div>
 +
    </div>
 +
 +
<div id="wrap">
 +
 +
<div class="prototypeprototype">
 +
 +
<br />
 +
<div class="fund">
 +
<h2 style="margin-top:30px; margin-bottom:10px; line-height: 24px;">We have a hardware, a software for Ios-i-GEM, yet …</h2><hr />
 +
 +
<p style="font-size:16px;">Besides the fungal killing switch and the functional prototype that help reduce concerns over GMO, we wonder what else in iGEM we can do as <b>social practice</b> to really engage in growers’ life and help them diminish threats posed by those pests. So far in our project, the entomogenous fungus provides a biological, no-harm attempt to eradicate the pests, which is one of the most important components of our idea. The prototype makes applying these genetically-engineered fungi practical and perhaps better. Now that we have a software and a hardware, what can we do more for the growers?</p>
 +
<p style="font-size:16px;">The answer came to the app, a well-design, thoughtful and realistic app. Growers, as well as government officials can simply check the “Taiwan Pest Prediction web”, where we provide 4 common pests in Taiwan with time scale from 1 day to 7 days. We offer the predicted pest scale from 0~16, 16~64, and above 64, 4 ranges in <i>B. dorsalis</i>. We convert prediction question into classification question, by building numerous classifiers and perform voting, we can get the prediction that most classifiers agree to. More important, we put all the code on github as open source, everyone and from every country can take advantage of our efforts to establish a better and powerful prediction program.</p>
 +
</div>
 +
 +
<div>
 +
<p style="font-size:16px;"><a href="http://taiwanpp.byethost8.com/web/backbone.html">Visit our website here</a></p>
 +
<p style="font-size:16px;"><a href="https://github.com/IandRover/TaiwanPP_FTP.git">Fork us from Gitbhub</a></p>
 +
<img src="https://static.igem.org/mediawiki/2016/5/50/T--NYMU-Taipei--photo-media-analysis-static_qr_code_without_logo.jpg" width="12%" />
 +
</div>
 +
 +
<div class="fund">
 +
<h2 style="margin-top:30px; margin-bottom:10px; line-height: 24px;">Collaboration</h2><hr />
 +
<p style="font-size:16px;">This app is a successful realization of our vision through the collaboration with NCTU. In August, after we have completed the web crawling part, we participated in the Asia-Pacific iGEM conference hosted by NCKU, and met our friends from NCKU. It was incredible to meet people with similar ideas and, most importantly, we decided to collaborate during the conference. From then on, we had several face-to-face talks in Facebook Messenger and frequent calls. </p>
 +
 +
<p style="font-size:16px;">The attribution is listed as following:</p>
 +
<h3 style="margin-top:30px; margin-bottom:10px; line-height: 20px;">NCTU</h3>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Provided us with expertise in FTP, web crawling, and concepts for the workflow</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Streamline code, making it more readable and understandable.</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Provide the idea of open source, and we did put our code on github.</p></li>
 +
<h3 style="margin-top:30px; margin-bottom:10px; line-height: 20px;">NYMU</h3>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">First come up with this idea</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Write program for web-crawling, date processing, machine learning and FTP uploading.</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Establish FTP host and write UI website.</p></li>
 +
</div>
 +
 +
<div class="fund">
 +
<h2 style="margin-top:30px; margin-bottom:10px; line-height: 24px;">How does it work?</h2>
 +
<hr>
 +
<div class="imageimage">
 +
<img src="https://static.igem.org/mediawiki/2016/d/d2/T--NYMU-Taipei--photo-media-analysis-%E5%9C%96%E7%89%874.png" width="100%" />
 +
</div>
 +
<p style="font-size:16px; white-space:pre-wrap; line-height: 24px;">Step 1: Update climatological data
 +
Connect to Agriculture weather website. Crawl and store the newest climatological data.
 +
Step 2: Data process and making prediction
 +
Grasp necessary information from html, and build up several SVM and RandomForest classifiers for prediction. Merge the outcome with the web frame.
 +
Step 3: upload to FTP
 +
Upload the html files to FTP.
 +
</p>
 +
</div>
 +
 +
<div class="fund">
 +
<p style="font-size:24px;">Data description:</p><hr />
 +
<img src="https://static.igem.org/mediawiki/2016/f/fe/T--NYMU-Taipei--photo-media-analysis-data-desciption-1.png" width="100%" />
 +
<p style="font-size:24px;">Hypothesis:</p><hr />
 +
<li style="list-style-type:square;"><p style="font-size:16px;">According to reports from Taiwan Agricultural Research Institute, the weather condition can directly or indirectly influence the maturation of pest. Take oriental fruit fly for example, days of raining can lead to the maturation. Low temperature, in contrast, resulting in small population size of oriental fruit fly. In building the model, we presumed that there is a relationship between weather and pest group size, such that there is a transformational matrix that transforms the weather information approximately to pest group size.</p></li>
 +
<li style="list-style-type:square;"><p style="font-size:16px;">Feature selection: We choose average day temperature, highest day temperature, lowest day temperature, rainfall as feature.</p></li>
 +
</ul>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Model selection:</p></li>
 +
 +
<p style="font-size:16px;">We are using two famous and frequently used model in building prediction model, SVM and RandomForest.</p>
 +
 +
<p style="font-size:16;"><b>Support Vector Machine</b></p>
 +
<img src="https://static.igem.org/mediawiki/2016/9/99/T--NYMU-Taipei--photo-main-page-TPP5.png" width="70%" />
 +
<p style="font-size:10px;">A Gentle Introduction to Support Vector Machines in Biomedicine, Alexander Statnikov*, Douglas Hardin# , Isabelle Guyon†, Constantin F. Aliferis<sup><a href="http://www.med.nyu.edu/chibi/sites/default/files/chibi/Final.pdf">[1]</a></sup></p>
 +
 +
<p style="font-size:16;"><b>Random Forest</b></p>
 +
<img src="https://static.igem.org/mediawiki/2016/a/ae/T--NYMU-Taipei--photo-main-page-TPP6.png" width="70%" />
 +
<p style="font-size:10px;">ICCV TUTORIAL, BOOSTING and Random Forest for Visual Recognition, Tae-Kyun Kim , Jamie Shotton , Björn Stenger<sup><a hred="http://www.iis.ee.ic.ac.uk/icvl/iccv09_tutorial.html">[2]</a></sup></p>
 +
 +
<p style="font-size:16px;">SVM is a machine learning method, by transforming data from dataspace into a hyper space, we can find a hyperplane that can separate these data with different labels. The SVM is trying to minimize the transformation matrix in some way. RandomForest is an ensembled decision forest. With building several decision trees, we are able to find out the best prediction that most decision tree agree.</p>
 +
<p style="font-size:16px;">In processing and testing data, member of NYMU discover that SVM made incredibly great prediction(~100%) when the datasets are big enough, while random forest got average score of 80%. However, in some cases, when randomforest model reached 60~70% of accuracy score, SVM got 50% or worse. We thus decide to build a simple ensembled prediction model, with different parameters and training datasets. And choose the pest population size that most classifiers agree. </p><br />
 +
<p style="font-size:20px;text-align:center;"><b>Additional reading about machine learning techiniques</b></p><hr />
 +
<ul>
 +
<li style="list-style-type:square;"><p style="font-size:16px; white-space:pre-wrap;">SVM
 +
<a hred="http://www.svm-tutorial.com/2014/11/svm-understanding-math-part-1/">A clear intro</a>
 +
<a hred="http://www.med.nyu.edu/chibi/sites/default/files/chibi/Final.pdf">SVM without tears</a>: page 19 is incredibly and clear
 +
</p></li>
 +
 +
<li style="list-style-type:square;"><p style="font-size:16px; white-space:pre-wrap;">RandomForest
 +
To realize RandomForest, it is recommended to <a href="https://en.wikipedia.org/wiki/Decision_tree">understand decision tree first</a>
 +
Here is <a hred="https://en.wikipedia.org/wiki/Random_forest">an intro to RandomForest In Wikipedia</a>
 +
<a hred="https://home.zhaw.ch/~dueo/bbs/files/random-forest-intro-presented.pdf">Deliberation on RandomForest</a>
 +
</p></li>
 +
</ul>
 +
 +
<p>icon credit: Freepik, Wissawa Khamsriwath, Madebyoliver, Gregor Cresnar</p></div>
 +
 +
<!--prototypeprototypesp--></div>
 +
 +
<div class="prototypeprototypesp">
 +
<br />
 +
<div class="fund">
 +
<h2 style="margin-top:10px; margin-bottom:10px; line-height: 24px;">We have a hardware, a software for Ios-i-GEM, yet …</h2><hr />
 +
 +
<p style="font-size:16px;">Besides the fungal killing switch and the functional prototype that help reduce concerns over GMO, we wonder what else in iGEM we can do as <b>social practice</b> to really engage in growers’ life and help them diminish threats posed by those pests. So far in our project, the entomogenous fungus provides a biological, no-harm attempt to eradicate the pests, which is one of the most important components of our idea. The prototype makes applying these genetically-engineered fungi practical and perhaps better. Now that we have a software and a hardware, what can we do more for the growers?</p>
 +
<p style="font-size:16px;">The answer came to the app, a well-design, thoughtful and realistic app. Growers, as well as government officials can simply check the “Taiwan Pest Prediction web”, where we provide 4 common pests in Taiwan with time scale from 1 day to 7 days. We offer the predicted pest scale from 0~16, 16~64, and above 64, 4 ranges in <i>B. dorsalis</i>. We convert prediction question into classification question, by building numerous classifiers and perform voting, we can get the prediction that most classifiers agree to. More important, we put all the code on github as open source, everyone and from every country can take advantage of our efforts to establish a better and powerful prediction program.</p>
 +
</div>
 +
 +
<div>
 +
<p style="font-size:16px;"><a href="http://taiwanpp.byethost8.com/web/backbone.html">Visit our website here</a></p>
 +
<p style="font-size:16px;"><a href="https://github.com/IandRover/TaiwanPP_FTP.git">Fork us from Gitbhub</a></p>
 +
<img src="https://static.igem.org/mediawiki/2016/5/50/T--NYMU-Taipei--photo-media-analysis-static_qr_code_without_logo.jpg" width="12%" />
 +
</div>
 +
 +
<div class="fund">
 +
<h2 style="margin-top:30px; margin-bottom:10px; line-height: 24px;">Collaboration</h2><hr />
 +
<p style="font-size:16px;">This app is the outcome of our collaboration with NCTU. We first developed our idea of construct such a web app for social practice. In August, when we have done the web crawling part, we participated in Asia-Pacific iGEM conference hosted by NCKU, and met our friends from NCKU. It was incredible to meet friends with similar ideas and, most importantly, we decided to collaborate at that time. From then, we had several face-to-face talks in FB Messenger and frequent calls. </p>
 +
 +
<p style="font-size:16px;">The attribution is listed as following:</p>
 +
<h3 style="margin-top:30px; margin-bottom:10px; line-height: 20px;">NCTU</h3>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Provide us with expertise in FTP, web crawling, and the concept in pipeline</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Streamline code, making it more readable and understandable.</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Provide the idea of open source, and we did put our code on github.</p></li>
 +
<h3 style="margin-top:30px; margin-bottom:10px; line-height: 20px;">NYMU</h3>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">First come up with this idea</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Write program for web-crawling, date processing, machine learning and FTP uploading.</p></li>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Establish FTP host and write UI website.</p></li>
 +
</div>
 +
 +
<div class="fund">
 +
<h2 style="margin-top:30px; margin-bottom:10px; line-height: 24px;">How does it work?</h2>
 +
<hr>
 +
<div class="imageimage">
 +
<img src="https://static.igem.org/mediawiki/2016/d/d2/T--NYMU-Taipei--photo-media-analysis-%E5%9C%96%E7%89%874.png" width="100%" />
 +
</div>
 +
<p style="font-size:16px; white-space:pre-wrap; line-height: 24px;">Step 1: Update climatological data
 +
Connect to Agriculture weather website. Crawl and store the newest climatological data.
 +
Step 2: Data process and making prediction
 +
Grasp necessary information from html, and build up several SVM and RandomForest classifiers for prediction. Merge the outcome with the web frame.
 +
Step 3: upload to FTP
 +
Upload the html files to FTP.
 +
</p>
 +
</div>
 +
 +
<div class="fund">
 +
<p style="font-size:24px;">Data description:</p><hr />
 +
<img src="https://static.igem.org/mediawiki/2016/f/fe/T--NYMU-Taipei--photo-media-analysis-data-desciption-1.png" width="100%" />
 +
<p style="font-size:24px;">Hypothesis:</p><hr />
 +
<li style="list-style-type:square;"><p style="font-size:16px;">According to reports from Taiwan Agricultural Research Institute, the weather condition can directly or indirectly influence the maturation of pest. Take oriental fruit fly for example, days of raining can lead to the maturation. Low temperature, in contrast, resulting in small population size of oriental fruit fly. In building the model, we presumed that there is a relationship between weather and pest group size, such that there is a transformational matrix that transforms the weather information approximately to pest group size.</p></li>
 +
<li style="list-style-type:square;"><p style="font-size:16px;">Feature selection: We choose average day temperature, highest day temperature, lowest day temperature, rainfall as feature.</p></li>
 +
</ul>
 +
<li style="list-style-type:disc;"><p style="font-size:16px;">Model selection:</p></li>
 +
 +
<p style="font-size:16px;">We are using two famous and frequently used model in building prediction model, SVM and RandomForest.</p>
 +
 +
<p style="font-size:16;"><b>Support Vector Machine</b></p>
 +
<img src="https://static.igem.org/mediawiki/2016/9/99/T--NYMU-Taipei--photo-main-page-TPP5.png" width="70%" />
 +
<p style="font-size:10px;">A Gentle Introduction to Support Vector Machines in Biomedicine, Alexander Statnikov*, Douglas Hardin# , Isabelle Guyon†, Constantin F. Aliferis<sup><a href="http://www.med.nyu.edu/chibi/sites/default/files/chibi/Final.pdf">[1]</a></sup></p>
 +
 +
<p style="font-size:16;"><b>Random Forest</b></p>
 +
<img src="https://static.igem.org/mediawiki/2016/a/ae/T--NYMU-Taipei--photo-main-page-TPP6.png" width="70%" />
 +
<p style="font-size:10px;">ICCV TUTORIAL, BOOSTING and Random Forest for Visual Recognition, Tae-Kyun Kim , Jamie Shotton , Björn Stenger<sup><a hred="http://www.iis.ee.ic.ac.uk/icvl/iccv09_tutorial.html">[2]</a></sup></p>
 +
 +
<p style="font-size:16px;">SVM is a machine learning method, by transforming data from dataspace into a hyper space, we can find a hyperplane that can separate these data with different labels. The SVM is trying to minimize the transformation matrix in some way. RandomForest is an ensembled decision forest. With building several decision trees, we are able to find out the best prediction that most decision tree agree.</p>
 +
<p style="font-size:16px;">In processing and testing data, member of NYMU discover that SVM made incredibly great prediction(~100%) when the datasets are big enough, while random forest got average score of 80%. However, in some cases, when randomforest model reached 60~70% of accuracy score, SVM got 50% or worse. We thus decide to build a simple ensembled prediction model, with different parameters and training datasets. And choose the pest population size that most classifiers agree. </p><br />
 +
<p style="font-size:20px;text-align:center;"><b>Additional reading about machine learning techiniques</b></p><hr />
 +
<ul>
 +
<li style="list-style-type:square;"><p style="font-size:16px; white-space:pre-wrap;">SVM
 +
<a hred="http://www.svm-tutorial.com/2014/11/svm-understanding-math-part-1/">A clear intro</a>
 +
<a hred="http://www.med.nyu.edu/chibi/sites/default/files/chibi/Final.pdf">SVM without tears</a>: page 19 is incredibly and clear
 +
</p></li>
 +
 +
<li style="list-style-type:square;"><p style="font-size:16px; white-space:pre-wrap;">RandomForest
 +
To realize RandomForest, it is recommended to <a href="https://en.wikipedia.org/wiki/Decision_tree">understand decision tree first</a>
 +
Here is <a hred="https://en.wikipedia.org/wiki/Random_forest">an intro to RandomForest In Wikipedia</a>
 +
<a hred="https://home.zhaw.ch/~dueo/bbs/files/random-forest-intro-presented.pdf">Deliberation on RandomForest</a>
 +
</p></li>
 +
</ul>
 +
 +
<p>icon credit: Freepik, Wissawa Khamsriwath, Madebyoliver, Gregor Cresnar</p>
 +
 +
<!--prototypeprototypesp--></div>
 +
 +
<!--wrap--></div>
 +
 +
</body>
 +
 +
</html>

Latest revision as of 22:30, 19 October 2016


We have a hardware, a software for Ios-i-GEM, yet …

Besides the fungal killing switch and the functional prototype that help reduce concerns over GMO, we wonder what else in iGEM we can do as social practice to really engage in growers’ life and help them diminish threats posed by those pests. So far in our project, the entomogenous fungus provides a biological, no-harm attempt to eradicate the pests, which is one of the most important components of our idea. The prototype makes applying these genetically-engineered fungi practical and perhaps better. Now that we have a software and a hardware, what can we do more for the growers?

The answer came to the app, a well-design, thoughtful and realistic app. Growers, as well as government officials can simply check the “Taiwan Pest Prediction web”, where we provide 4 common pests in Taiwan with time scale from 1 day to 7 days. We offer the predicted pest scale from 0~16, 16~64, and above 64, 4 ranges in B. dorsalis. We convert prediction question into classification question, by building numerous classifiers and perform voting, we can get the prediction that most classifiers agree to. More important, we put all the code on github as open source, everyone and from every country can take advantage of our efforts to establish a better and powerful prediction program.

Collaboration

This app is a successful realization of our vision through the collaboration with NCTU. In August, after we have completed the web crawling part, we participated in the Asia-Pacific iGEM conference hosted by NCKU, and met our friends from NCKU. It was incredible to meet people with similar ideas and, most importantly, we decided to collaborate during the conference. From then on, we had several face-to-face talks in Facebook Messenger and frequent calls.

The attribution is listed as following:

NCTU

  • Provided us with expertise in FTP, web crawling, and concepts for the workflow

  • Streamline code, making it more readable and understandable.

  • Provide the idea of open source, and we did put our code on github.

    • NYMU

  • First come up with this idea

  • Write program for web-crawling, date processing, machine learning and FTP uploading.

  • Establish FTP host and write UI website.

    • How does it work?

    Step 1: Update climatological data Connect to Agriculture weather website. Crawl and store the newest climatological data. Step 2: Data process and making prediction Grasp necessary information from html, and build up several SVM and RandomForest classifiers for prediction. Merge the outcome with the web frame. Step 3: upload to FTP Upload the html files to FTP.

    Data description:

    Hypothesis:

  • According to reports from Taiwan Agricultural Research Institute, the weather condition can directly or indirectly influence the maturation of pest. Take oriental fruit fly for example, days of raining can lead to the maturation. Low temperature, in contrast, resulting in small population size of oriental fruit fly. In building the model, we presumed that there is a relationship between weather and pest group size, such that there is a transformational matrix that transforms the weather information approximately to pest group size.

  • Feature selection: We choose average day temperature, highest day temperature, lowest day temperature, rainfall as feature.

  • Model selection:

    • We are using two famous and frequently used model in building prediction model, SVM and RandomForest.

    Support Vector Machine

    A Gentle Introduction to Support Vector Machines in Biomedicine, Alexander Statnikov*, Douglas Hardin# , Isabelle Guyon†, Constantin F. Aliferis[1]

    Random Forest

    ICCV TUTORIAL, BOOSTING and Random Forest for Visual Recognition, Tae-Kyun Kim , Jamie Shotton , Björn Stenger[2]

    SVM is a machine learning method, by transforming data from dataspace into a hyper space, we can find a hyperplane that can separate these data with different labels. The SVM is trying to minimize the transformation matrix in some way. RandomForest is an ensembled decision forest. With building several decision trees, we are able to find out the best prediction that most decision tree agree.

    In processing and testing data, member of NYMU discover that SVM made incredibly great prediction(~100%) when the datasets are big enough, while random forest got average score of 80%. However, in some cases, when randomforest model reached 60~70% of accuracy score, SVM got 50% or worse. We thus decide to build a simple ensembled prediction model, with different parameters and training datasets. And choose the pest population size that most classifiers agree.


    Additional reading about machine learning techiniques

    icon credit: Freepik, Wissawa Khamsriwath, Madebyoliver, Gregor Cresnar


    We have a hardware, a software for Ios-i-GEM, yet …

    Besides the fungal killing switch and the functional prototype that help reduce concerns over GMO, we wonder what else in iGEM we can do as social practice to really engage in growers’ life and help them diminish threats posed by those pests. So far in our project, the entomogenous fungus provides a biological, no-harm attempt to eradicate the pests, which is one of the most important components of our idea. The prototype makes applying these genetically-engineered fungi practical and perhaps better. Now that we have a software and a hardware, what can we do more for the growers?

    The answer came to the app, a well-design, thoughtful and realistic app. Growers, as well as government officials can simply check the “Taiwan Pest Prediction web”, where we provide 4 common pests in Taiwan with time scale from 1 day to 7 days. We offer the predicted pest scale from 0~16, 16~64, and above 64, 4 ranges in B. dorsalis. We convert prediction question into classification question, by building numerous classifiers and perform voting, we can get the prediction that most classifiers agree to. More important, we put all the code on github as open source, everyone and from every country can take advantage of our efforts to establish a better and powerful prediction program.

    Collaboration

    This app is the outcome of our collaboration with NCTU. We first developed our idea of construct such a web app for social practice. In August, when we have done the web crawling part, we participated in Asia-Pacific iGEM conference hosted by NCKU, and met our friends from NCKU. It was incredible to meet friends with similar ideas and, most importantly, we decided to collaborate at that time. From then, we had several face-to-face talks in FB Messenger and frequent calls.

    The attribution is listed as following:

    NCTU

  • Provide us with expertise in FTP, web crawling, and the concept in pipeline

  • Streamline code, making it more readable and understandable.

  • Provide the idea of open source, and we did put our code on github.

    • NYMU

  • First come up with this idea

  • Write program for web-crawling, date processing, machine learning and FTP uploading.

  • Establish FTP host and write UI website.

    • How does it work?

    Step 1: Update climatological data Connect to Agriculture weather website. Crawl and store the newest climatological data. Step 2: Data process and making prediction Grasp necessary information from html, and build up several SVM and RandomForest classifiers for prediction. Merge the outcome with the web frame. Step 3: upload to FTP Upload the html files to FTP.

    Data description:

    Hypothesis:

  • According to reports from Taiwan Agricultural Research Institute, the weather condition can directly or indirectly influence the maturation of pest. Take oriental fruit fly for example, days of raining can lead to the maturation. Low temperature, in contrast, resulting in small population size of oriental fruit fly. In building the model, we presumed that there is a relationship between weather and pest group size, such that there is a transformational matrix that transforms the weather information approximately to pest group size.

  • Feature selection: We choose average day temperature, highest day temperature, lowest day temperature, rainfall as feature.

  • Model selection:

    • We are using two famous and frequently used model in building prediction model, SVM and RandomForest.

    Support Vector Machine

    A Gentle Introduction to Support Vector Machines in Biomedicine, Alexander Statnikov*, Douglas Hardin# , Isabelle Guyon†, Constantin F. Aliferis[1]

    Random Forest

    ICCV TUTORIAL, BOOSTING and Random Forest for Visual Recognition, Tae-Kyun Kim , Jamie Shotton , Björn Stenger[2]

    SVM is a machine learning method, by transforming data from dataspace into a hyper space, we can find a hyperplane that can separate these data with different labels. The SVM is trying to minimize the transformation matrix in some way. RandomForest is an ensembled decision forest. With building several decision trees, we are able to find out the best prediction that most decision tree agree.

    In processing and testing data, member of NYMU discover that SVM made incredibly great prediction(~100%) when the datasets are big enough, while random forest got average score of 80%. However, in some cases, when randomforest model reached 60~70% of accuracy score, SVM got 50% or worse. We thus decide to build a simple ensembled prediction model, with different parameters and training datasets. And choose the pest population size that most classifiers agree.


    Additional reading about machine learning techiniques

    icon credit: Freepik, Wissawa Khamsriwath, Madebyoliver, Gregor Cresnar