(1)The remaster branch is doing web crawling tutorgroup, ectutor, l4tutor
(2) Need to separate the config and model ,using bash csv parser
- sudo apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules
- Then take reference from: http://askubuntu.com/questions/522431/how-to-send-an-email-using-command-line
- create a "config.csv" file, the sample format can be seen as the table below.
- install "awk-csv-parser": https://github.com/geoffroy-aubry/awk-csv-parser (remember to add the alias as prompted in the installation guide)
- install "ant": by "sudo apt-get install ant"
- Then set "crobtab -e" to run TutorNotification.sh, if want to run the job periodically Example: append "30 9-23 * * * /bin/bash /abs/path/to/working/path/TutorNotification.sh" to your cron config
**Note** the first two steps are to enable you to send email by cmd line, you don't need it if you know you have already
Sample of config file
====
| WebKey | Type | Value |
|---|---|---|
| "" | WC_RECIPIENT | "sample1@sample.com" |
| "" | WC_RECIPIENT | "sample2@sample.com" |
| "ECTutor" | WC_SEARCH_CRIT | "物理" |
| "ECTutor" | WC_SEARCH_CRIT | "數學" |
| "ECTutor" | WC_SEARCH_CRIT | "生物" |
| "ECTutor" | WC_SEARCH_CRIT | "化學" |
| "ECTutor" | WC_SEARCH_CRIT | "科學" |
| "ECTutor" | WC_SEARCH_CRIT | "Math" |
| "ECTutor" | WC_SEARCH_CRIT | "math" |
| "ECTutor" | WC_SEARCH_CRIT | "Phy" |
| "ECTutor" | WC_SEARCH_CRIT | "phy" |
| "ECTutor" | WC_SEARCH_CRIT | "Chem" |
| "ECTutor" | WC_SEARCH_CRIT | "chem" |
| "ECTutor" | WC_SEARCH_CRIT | "M1" |
| "ECTutor" | WC_SEARCH_CRIT | "m1" |
| "ECTutor" | WC_SEARCH_CRIT | "M2" |
| "ECTutor" | WC_SEARCH_CRIT | "m2" |
| "ECTutor" | WC_SEARCH_CRIT | "Bio" |
| "ECTutor" | WC_SEARCH_CRIT | "bio" |
| "ECTutor" | WC_SEARCH_CRIT | "Science" |
| "ECTutor" | WC_SEARCH_CRIT | "science" |
| "ECTutor" | WC_SEARCH_CRIT | "IB" |
| "ECTutor" | WC_SEARCH_CRIT | "ib" |
| "ECTutor" | WC_SEARCH_CRIT | "GCSE" |
| "ECTutor" | WC_SEARCH_CRIT | "GCSE" |
| "ECTutor" | WC_URL | "http://www.ectutor.com/popup\_case.php?id=" |
| "ECTutor" | WC_SEARCH_COND_PRICE_ABOVE | 130 |
| "ECTutor" | WC_INDEX_URL | "http://www.ectutor.com/search.php?maxPage=10&cPage=1&infoType=2&infoSex=&subject\_1=&district\_1=&educ\_1=&tutor\_id=" |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 觀塘 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 銅鑼灣 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 小西灣 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 沙田 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 將軍澳 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 馬鞍山 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 藍田 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 柴灣 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 北角 |
| "ECTutor" | WC_SEARCH_OUT_CRIT | 鴨脷洲 |
| "L4Tutor" | WC_SEARCH_CRIT | "物理" |
| "L4Tutor" | WC_SEARCH_CRIT | "數學" |
| "L4Tutor" | WC_SEARCH_CRIT | "生物" |
| "L4Tutor" | WC_SEARCH_CRIT | "化學" |
| "L4Tutor" | WC_SEARCH_CRIT | "科學" |
| "L4Tutor" | WC_SEARCH_CRIT | "Math" |
| "L4Tutor" | WC_SEARCH_CRIT | "math" |
| "L4Tutor" | WC_SEARCH_CRIT | "Phy" |
| "L4Tutor" | WC_SEARCH_CRIT | "phy" |
| "L4Tutor" | WC_SEARCH_CRIT | "Chem" |
| "L4Tutor" | WC_SEARCH_CRIT | "chem" |
| "L4Tutor" | WC_SEARCH_CRIT | "M1" |
| "L4Tutor" | WC_SEARCH_CRIT | "m1" |
| "L4Tutor" | WC_SEARCH_CRIT | "M2" |
| "L4Tutor" | WC_SEARCH_CRIT | "m2" |
| "L4Tutor" | WC_SEARCH_CRIT | "Bio" |
| "L4Tutor" | WC_SEARCH_CRIT | "bio" |
| "L4Tutor" | WC_SEARCH_CRIT | "Science" |
| "L4Tutor" | WC_SEARCH_CRIT | "science" |
| "L4Tutor" | WC_SEARCH_CRIT | "IB" |
| "L4Tutor" | WC_SEARCH_CRIT | "ib" |
| "L4Tutor" | WC_SEARCH_CRIT | "GCSE" |
| "L4Tutor" | WC_SEARCH_CRIT | "GCSE" |
| "L4Tutor" | WC_URL | "http://www.looking4tutor.com/popup\_case.php?id=" |
| "L4Tutor" | WC_SEARCH_COND_PRICE_ABOVE | 130 |
| "L4Tutor" | WC_INDEX_URL | "http://looking4tutor.com/search.php?maxPage=10&cPage=1&infoType=2&infoSex=&education\_1=&district\_1=&subject\_1=&tutor\_id=" |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 觀塘 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 銅鑼灣 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 小西灣 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 沙田 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 將軍澳 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 馬鞍山 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 藍田 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 柴灣 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 北角 |
| "L4Tutor" | WC_SEARCH_OUT_CRIT | 鴨脷洲 |
| "TutorGroup" | WC_SEARCH_CRIT | "物理" |
| "TutorGroup" | WC_SEARCH_CRIT | "數學" |
| "TutorGroup" | WC_SEARCH_CRIT | "生物" |
| "TutorGroup" | WC_SEARCH_CRIT | "化學" |
| "TutorGroup" | WC_SEARCH_CRIT | "科學" |
| "TutorGroup" | WC_SEARCH_CRIT | "Math" |
| "TutorGroup" | WC_SEARCH_CRIT | "math" |
| "TutorGroup" | WC_SEARCH_CRIT | "Phy" |
| "TutorGroup" | WC_SEARCH_CRIT | "phy" |
| "TutorGroup" | WC_SEARCH_CRIT | "Chem" |
| "TutorGroup" | WC_SEARCH_CRIT | "chem" |
| "TutorGroup" | WC_SEARCH_CRIT | "M1" |
| "TutorGroup" | WC_SEARCH_CRIT | "m1" |
| "TutorGroup" | WC_SEARCH_CRIT | "M2" |
| "TutorGroup" | WC_SEARCH_CRIT | "m2" |
| "TutorGroup" | WC_SEARCH_CRIT | "Bio" |
| "TutorGroup" | WC_SEARCH_CRIT | "bio" |
| "TutorGroup" | WC_SEARCH_CRIT | "Science" |
| "TutorGroup" | WC_SEARCH_CRIT | "science" |
| "TutorGroup" | WC_SEARCH_CRIT | "IB" |
| "TutorGroup" | WC_SEARCH_CRIT | "ib" |
| "TutorGroup" | WC_SEARCH_CRIT | "GCSE" |
| "TutorGroup" | WC_SEARCH_CRIT | "GCSE" |
| "TutorGroup" | WC_URL | "http://tutorgroup.hk/tutor/index.php" |
| "TutorGroup" | WC_URL | "http://tutorgroup.hk/tutor/index.php?page=2" |
| "TutorGroup" | WC_SEARCH_COND_PRICE_ABOVE | 130 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 觀塘 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 銅鑼灣 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 小西灣 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 沙田 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 將軍澳 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 馬鞍山 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 藍田 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 柴灣 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 北角 |
| "TutorGroup" | WC_SEARCH_OUT_CRIT | 鴨脷洲 |
**Note** at WC\_SEARCH\_COND\_PRICE\_ABOVE, value -1 is reserved
Other
====
* Make sure your system date is correct, synced
* Make sure your mail system is operatin