2. Instructions & Brief
Task A
We will analyse the top emoticons found in the messages of tweets, from the ‘msgraw_sample.txt’ data used in the tutorial of Week 7. Note this should be done a Linux machine or similar where bash supported.
Task A.1 (4 marks)
The first sub-task is to extract the top 20 emoticons and their counts from the tweets. This must not be done entirely manually, and it can only be done using a single shell script. So you need to write a single shell script ‘tweet2emo.sh’ that will input ‘msgraw_sample.txt’ from stdin and produce a CSV file ‘potential_emoticon.csv’ giving a list of candidate emoticons with their occurrence counts. The important word here is “candidate”. Perhaps only 1 in 5 of your candidates are emoticons. Then you need to edit this by hand, deleting non-emoticons, and deleting less frequent ones, to get your final, list ’emoticon.csv’.
So for this task, you must submit:
(1) a single bash script, ‘tweet2emo.sh’ : this must output, one per line, a candidate emoticon and a count of occurrence, and cannot have any Python or R programmes embedded in it. More details on how to do this below.
(2) the candidate list of emoticons generated by the script, ‘potential_emoticon.csv’ : CSV file, TAB delimited file with (count, text-emoticon).
(3) the final list of emoticons selected, ’emoticon.csv’ : CSV file, TAB delimited file with (count, text-emoticon); these should be the 20 most frequent emoticons from ‘potential_emoticon.csv’, but you will have to select yourself, manually by editing, which are actually emoticons. To do this, you may use an externally provided list of recognised emoticons, but not should be used in step (2).
(4) a description for this task is included in your final PDF report describing the method used for the bash script, and then the method used to edit the file, to get the file for step (3).
Your bash scripts might take 2-5-10 lines and might require storing intermediate files.
The following single line commands, which process a file from stdin and generate stdout should be useful for this task:
perl -p -e ‘s/s+/n/g;’
— tokenise each line of text by converting space characters to newlines;
NOTE: this reportedly also work on Windows where newline character is different
perl -p -e ‘s/>/>/g; s/</
— convert embedded HTML escapes for ‘>’ and ‘
— you need to do this if you want to capture emoticons using the ‘<‘ or the ‘>’ characters, like ‘
sort | uniq -c | perl -p -e ‘s/^s+//; s/ /t/; ‘
— assumes the input file has one item per line
— sort and count the items and generates TAB delimited file with (count, item) entries
Specially, in order to recognise potential emoticons, you will need to write suitable greps. Here are some examples:
grep -e ‘^_^’
— match lines containing the string “^_^”
grep -e ‘^^_^’
— match lines starting with the string “^_^”, the initial “^”, called an anchor, says match start of line
grep -e ‘^_^$’
— match lines ending with the string “^_^”, the final “$”, called an anchor, says match end of line
grep -e ‘^^_^$’
— match lines made exactly of the string “^_^”, using beginning and ending anchors
grep -e ‘^0_0$’
— match lines made exactly of the string “0_0”
grep -e ‘^^_^$’ -e ‘^0_0$’
— match lines made exactly of the string “^_^” or “0_0”; so two match strings are ORed
grep -e ‘^[.:^]$’
— match lines made exactly of the characters in the set “.:^”
— the construction “[ … ]” means “characters in the set ” … ” but be warned some characters used inside have strange effects, like “-“, see next
grep -e ‘^[0-9ABC]$’
— match lines made exactly of the digits (“0-9” means in the range “0” to “9”) or characters “ABC”
grep -e ‘^[-0-9ABC]$’
— match lines made exactly of the dash “-“, the digits, or the characters “ABC”
— we place “-” at the front to stop in meaning “range”
For more detail on grep see:
https://opensourceforu.com/2012/06/beginners-guide-gnu-grep-basics-regular-expressions/
But my advice is “keep it simple” and stick with the above constructs. Remember you get to edit the final results by hand anyway. But if your grep match strings say “7” is an emoticon, it probably isn’t a strong enough filter.
Task A.2 (4 marks)
We would like to compute word co-occurrence with emoticons. So suppose we have the tweet:
loved the results of the game 😉
then this means that emoticon ‘;-)’ co-occurs once with each of the words in the list ‘ loved the results of the game’ once.
You can use the supplied Python program ’emoword.py” which uses a single emoticon, takes ‘msgraw_sample.txt’ as stdin and outputs a raw list of co-occurring tokens.
./emoword.py ‘:))’
Note the emoticon is enclosed in single quotes because the punctuation can cause bash to do weird things otherwise.
You can also put this in a bash loop to run over your emoticon list like so:
for E in ‘;)’ ‘:)’ ‘echo running this emoticon $E
done
or counting them too using
CNT=1
for E in ‘;)’ ‘:)’ ‘echo running this emoticon $E > $CNT.out
CNT=$(( $CNT + 1)) # this is arithmetic in bash
done
But be warned, bash does strange things with punctuation … it treats it differently as it plays a role in the language. So while you can have a loop doing this:
for E in ‘;)’ ‘:)’ ‘
where you have edited in your emoticons, and used the single quotes to tell bash the quoted text is a single token, if instead you try and be clever and read them from a file
for E in `cat emoticons.txt` ; do
then bash well see individual punctuation and probably fail to work in the way you want.
For each emoticon in your list ’emoticon.csv’, find a list of the 10-20 most commonly occurring interesting words. Report on these words in your final PDF report. Note that words like “the” and “in” are called stop words, see https://en.wikipedia.org/wiki/Stop_words, and are uninteresting, so try and exclude these from your report.
So for this task, you must submit:
(1) a single bash script, ’emowords.sh’ : as used to support your answers, perhaps calling ’emoword.py’; this should output for each of your 20 emoticons the most frequent words co-occurring with it (in tweets); use what ever format suits, as the results will be transferred and written up in your report.
(2) a description for this task is included in your final PDF report describing the method used for the bash script, and then the final list of selected interesting words per emoticon, and how you got them.
Task A.3 (2 marks)
See if there are other interesting information you can get about these emoticons. For instance is there anything about countries/cities and emoticons? Which emoticons have long or short messages? Whats sorts of messages are attached to different emoticons?
You can use the Python program ’emodata.py” which reads your ’emoticon.csv’ file, takes ‘msgraw_sample.txt’ as stdin and outputs selected data from the tweet file.
./emodata.py
Report on this in your final PDF report. Use any technique or coding you like to get this information. Your report should describe what you did and your results.
Task B
Consider the two files ‘training.csv’ and ‘test.csv’.
Task B.1 (2 marks)
Plot histograms of X1, X2, X3 and X4 in train.csv respectively and answer: which variable(s) is(are) most likely samples drawn from normal distributions?
Task B.2 (4 marks)
Fit two linear regression models using train.csv.
Model 1: Y~X1+X2+X3+X4
Model 2: Y~X2+X3+X4
Which model has higher Multiple R-squared value?
Task B3 (4 marks)
Now use the coefficients of Model 1 and 2 respectively to predict the Y values of test.csv, then calculate the Mean Squared Errors (MSE) between the predictions and the true values. Which model has smaller MSE? Which model is better? More complex models always have higher R square but are they always better?
3. Assessment Criteria
The work required to prepare data, explore data and explain your findings should be all your own. If you use resources elsewhere, make sure that you acknowledge all of them in your PDF report. You may need to review the FIT citation styletutorial to make yourself familiar with appropriate citing and referencing for this assessment. Also, review the demystifying citing and referencingfor help.
The following outlines the criteria which you will be assessed against.
3.1 Grading Rubric
The following outlines the criteria which you will be assessed against:
- Ability to read data files and process them using bash and R commands.
- Ability to wrangle and process data into the required formats.
- Ability to use various graphical and non-graphical tools for performing exploratory data analysis and visualisation
- Ability to use basic tools for managing and processing big data;
- Ability to communicate your findings in your report.
The marks are allocated as follows:
- Task 1: 10 (=5% of total)
- Task 2: 10 (=5% of total)
3.2 Penalties
- Late submission: for all assessment items handed in after the official due date, and without an agreed extension, a 5% penalty applies to the student’s mark for each day after the due date (including weekends, and public holidays) for up to 7 days. Assessment items handed in after 7 days will not be considered.
- Word limit: There are no firm wording limits on the report. However, as a general guidance, it should not be exceeding 1000 words, excluding other supplementary materials (e.g., slides and transcript). Lengthy reports (i.e., over 1000 words) may incur a loss of mark due to the time limit a marker will spend on the report reading. For instance, they may only read the part within 1000 words and omit rest of the report. Notice that references consisting of URLs can be given at the end of the entry and are not included in the word count.
4. How to Submit
Once you have completed your work, take the following steps to submit your work.
- Include the following materials in your submission:
- A report in PDF containing your answers to all the questions.
- You can use Word or other word processing software to format your submission. Just save the final copy to a PDF before submitting.
- Make sure to include in the PDF screenshots/images of any graphs you generate in order to justify your answers to all the questions for both parts A and B.
- Two bash scripts so named containing the code to complete Task A.1 and task A.2.
- The two CSV files ‘potential_emoticon.csv’ and ’emoticon.csv’
- A Jupyter notebook file or plain text R file containing the R code you write to prepare and plot the data for Task B.
- If you’ve chosen other tools, please include the process of data preparation in your report as an appendix included with the PDF.
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Draw An Isometric Of The Wood Pellet Stove Shown 2848173
/in Uncategorized /by developer2. DRAW AN ISOMETRIC OF THE WOOD PELLET STOVE SHOWN.
Attachments:
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Does Tiger Brands Pursue A Cost Leadership Differentiation Or Focus Strategy Evalu 2816033
/in Uncategorized /by developer2.Does Tiger Brands pursue a cost leadership, Differentiation or focus Strategy? Evaluate its strategic approach in comparison to its competitors.
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Evaluate The Following Integrals Find F T Iff T 2nrof W Emtdw And F W Mm P6 W 4 Jw 3295971
/in Uncategorized /by developer2. Evaluate the following integrals: Find f(t) iff(t)=2nroF(w)eMtdw and F(w)=mm)p6(w). 4+jw 9+jw 3.
Attachments:
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Er Diagram Is Based On The Business Rules And Should Clearly Label All Entities En 2849787
/in Uncategorized /by developer2. ER diagram is based on the business rules, and should clearly label all entities, entity attributes, primary and foreign keys, relationship and connectivity. The cardinality is optional.Instruction: Use professional software (e.g., MS Office Visio) to draw the ER diagram. Crow’s Foot notation is preferable.(40 marks)3. Develop relational schemas. Relational schemas should be derived using the ERD. You should map cardinalities correctly from ERD to Relational Schema. You should clearly indicate the referential integrity constraints (primary and foreign key relationships) using arrows. Clearly indicate datatype for each attribute.e.g.Emp( eid: integer, ename: string(50), address: string(100), did: number)Dept(did: number, dname : string (15)) 4. SQL commands1. Create all tables in Deakin Oracle DBMS (about eight tables including compositetables) and Populate the tables with sample data (10 records in each table isrecommended).2. Alter the student table and add new field Date of Birth for student table. Type fordate of birth should be date.3. Increase the annual salary for all staff member by 5%.4. List the course numbers, course names a student who is doing computer sciencemajor could enrol for.5. Find the students with age between 18 and 21. Print their student number, nameand the age with the major.6. Create your own query. It must include a nested query. Submit the following:i question your query is answering the SQL queryii the mark for this question will depend on the complexity of the query.iii higher marks will be given for queries that are more complex and/or innovative.(15 marks)iv if you do not provide a description of what question the query is answering, you will get zero for this query.
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Expected Utility Theory An Individual Goes To The Store To Buy A New Iclicker For 3300681
/in Uncategorized /by developer2. Expected Utility Theory An individual goes to the store to buy a new iClicker for $40. The clerk at the store tells the individual that the same iClicker is on sale for $20 across campus. The individual goes to the other store. The same individual goes to the store to buy a new computer for $600. The clerk at the store tells the individual the same computer is on sale at the same store across campus for $580. The student does not go. Is this consistent with expected utility theory? Why or why not
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Final Project Business Plan Resources Appendix A Due Date Sunday Day 7 Individual 1039231
/in Uncategorized /by developer2. Final Project: Business PlanResources: Appendix ADue Date: Sunday Day 7 [Individual] forumWrite a 700- to 1,050-word paper, using APA guidelines, based on the scenario below.You want to start your own business. You found an investment group that is willing to give you the capital needed for the first year of your business, but only if you can convince them you have a solid plan for the success of this business. Your investor is very concerned with how the accounting functions of this business will be handled. You must persuade your investor to put up the capital by addressing the following questions in your business plan:· What is the name of your business?· What type of business structure is it (sole proprietorship, partnership, or corporation)?· Why did you choose that structure?· What type of services or products does your business provide?· What role will accounting play in the start up of your business?· What type of work characteristics will you look for when hiring your accounting staff?· What education should a person have in budgeting, internal controls, and cash management before going into business?· What kinds of internal controls will you put in place for the business?· How will your managers use financial information to predict outcomes for your business?I really do not know where to start and was wondering if anyone can give me a way to go?
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Glenn Foreman President Of Oceanview Development Corporation Is Considering Submi 2839863
/in Uncategorized /by developer(2) Glenn Foreman, president of Oceanview Development Corporation, is considering submitting a bid to purchase property that will be sold by sealed bid at a county tax foreclosure. Glenn’s initial judgment is to submit a bid of $5.5 million. Based on his experience, Glenn estimates that a bid of $5.5 million will have a 0.2 probability of being the highest bid and securing the property for Oceanview.
The sealed-bid procedure requires the bid to be submitted with a certified check for
8% of the amount bid. If the bid is rejected, the deposit is refunded. If the bid is accepted, the deposit is the down payment for the property. However, if the bid is accepted and the bidder does not follow through with the purchase and meet the remainder of the financial obligation within six months, the deposit will be forfeited. In this case, the county will offer the property to the next highest bidder.
If Oceanview’s bid is accepted and they obtain the property, the firm has to decide between building and selling a complex of luxury condominiums or building and selling a complex of single family residences. However, a complicating factor is that the property is currently zoned for single-family residences only. Glenn believes that a referendum could be placed on the voting ballot in time for the November election. Passage of the referendum would change the zoning of the property and permit construction of the condominiums. To determine whether Oceanview should submit the $5 million bid, Glenn conducted some preliminary analysis. This preliminary work provided an assessment of a 40% the probability that the referendum for a zoning change will be approved.
Cost and Revenue Estimates- Condominiums
Revenue from condominium sales: $16,000,000 with a probability of 75% or 7,000,000 with a probability of 25%
Cost: Construction expenses of $7,500,000
Cost and Revenue Estimates- Single Family Homes
Revenue from home sales: $12,000,000 with a probability of 70% or 6,000,000 with a probability of 30%
Cost: Construction expenses of $6,000,000
a. Draw the decision tree for this problem.
b. Calculate the expected values and determine the best decision for Oceanview
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 From An Historical Perspective How Has The Labor Market Experience Of Black Women 3297883
/in Uncategorized /by developer2. From an historical perspective, how has the labor market experience of black women and white women differed?
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Instructions Amp Brief Task A We Will Analyse The Top Emoticons Found In The Messa 2732785
/in Uncategorized /by developer2. Instructions & Brief
Task A
We will analyse the top emoticons found in the messages of tweets, from the ‘msgraw_sample.txt’ data used in the tutorial of Week 7. Note this should be done a Linux machine or similar where bash supported.
Task A.1 (4 marks)
The first sub-task is to extract the top 20 emoticons and their counts from the tweets. This must not be done entirely manually, and it can only be done using a single shell script. So you need to write a single shell script ‘tweet2emo.sh’ that will input ‘msgraw_sample.txt’ from stdin and produce a CSV file ‘potential_emoticon.csv’ giving a list of candidate emoticons with their occurrence counts. The important word here is “candidate”. Perhaps only 1 in 5 of your candidates are emoticons. Then you need to edit this by hand, deleting non-emoticons, and deleting less frequent ones, to get your final, list ’emoticon.csv’.
So for this task, you must submit:
(1) a single bash script, ‘tweet2emo.sh’ : this must output, one per line, a candidate emoticon and a count of occurrence, and cannot have any Python or R programmes embedded in it. More details on how to do this below.
(2) the candidate list of emoticons generated by the script, ‘potential_emoticon.csv’ : CSV file, TAB delimited file with (count, text-emoticon).
(3) the final list of emoticons selected, ’emoticon.csv’ : CSV file, TAB delimited file with (count, text-emoticon); these should be the 20 most frequent emoticons from ‘potential_emoticon.csv’, but you will have to select yourself, manually by editing, which are actually emoticons. To do this, you may use an externally provided list of recognised emoticons, but not should be used in step (2).
(4) a description for this task is included in your final PDF report describing the method used for the bash script, and then the method used to edit the file, to get the file for step (3).
Your bash scripts might take 2-5-10 lines and might require storing intermediate files.
The following single line commands, which process a file from stdin and generate stdout should be useful for this task:
perl -p -e ‘s/s+/n/g;’
— tokenise each line of text by converting space characters to newlines;
NOTE: this reportedly also work on Windows where newline character is different
perl -p -e ‘s/>/>/g; s/</
— convert embedded HTML escapes for ‘>’ and ‘
— you need to do this if you want to capture emoticons using the ‘<‘ or the ‘>’ characters, like ‘
sort | uniq -c | perl -p -e ‘s/^s+//; s/ /t/; ‘
— assumes the input file has one item per line
— sort and count the items and generates TAB delimited file with (count, item) entries
Specially, in order to recognise potential emoticons, you will need to write suitable greps. Here are some examples:
grep -e ‘^_^’
— match lines containing the string “^_^”
grep -e ‘^^_^’
— match lines starting with the string “^_^”, the initial “^”, called an anchor, says match start of line
grep -e ‘^_^$’
— match lines ending with the string “^_^”, the final “$”, called an anchor, says match end of line
grep -e ‘^^_^$’
— match lines made exactly of the string “^_^”, using beginning and ending anchors
grep -e ‘^0_0$’
— match lines made exactly of the string “0_0”
grep -e ‘^^_^$’ -e ‘^0_0$’
— match lines made exactly of the string “^_^” or “0_0”; so two match strings are ORed
grep -e ‘^[.:^]$’
— match lines made exactly of the characters in the set “.:^”
— the construction “[ … ]” means “characters in the set ” … ” but be warned some characters used inside have strange effects, like “-“, see next
grep -e ‘^[0-9ABC]$’
— match lines made exactly of the digits (“0-9” means in the range “0” to “9”) or characters “ABC”
grep -e ‘^[-0-9ABC]$’
— match lines made exactly of the dash “-“, the digits, or the characters “ABC”
— we place “-” at the front to stop in meaning “range”
For more detail on grep see:
https://opensourceforu.com/2012/06/beginners-guide-gnu-grep-basics-regular-expressions/
But my advice is “keep it simple” and stick with the above constructs. Remember you get to edit the final results by hand anyway. But if your grep match strings say “7” is an emoticon, it probably isn’t a strong enough filter.
Task A.2 (4 marks)
We would like to compute word co-occurrence with emoticons. So suppose we have the tweet:
loved the results of the game 😉
then this means that emoticon ‘;-)’ co-occurs once with each of the words in the list ‘ loved the results of the game’ once.
You can use the supplied Python program ’emoword.py” which uses a single emoticon, takes ‘msgraw_sample.txt’ as stdin and outputs a raw list of co-occurring tokens.
./emoword.py ‘:))’
Note the emoticon is enclosed in single quotes because the punctuation can cause bash to do weird things otherwise.
You can also put this in a bash loop to run over your emoticon list like so:
for E in ‘;)’ ‘:)’ ‘echo running this emoticon $E
done
or counting them too using
CNT=1
for E in ‘;)’ ‘:)’ ‘echo running this emoticon $E > $CNT.out
CNT=$(( $CNT + 1)) # this is arithmetic in bash
done
But be warned, bash does strange things with punctuation … it treats it differently as it plays a role in the language. So while you can have a loop doing this:
for E in ‘;)’ ‘:)’ ‘
where you have edited in your emoticons, and used the single quotes to tell bash the quoted text is a single token, if instead you try and be clever and read them from a file
for E in `cat emoticons.txt` ; do
then bash well see individual punctuation and probably fail to work in the way you want.
For each emoticon in your list ’emoticon.csv’, find a list of the 10-20 most commonly occurring interesting words. Report on these words in your final PDF report. Note that words like “the” and “in” are called stop words, see https://en.wikipedia.org/wiki/Stop_words, and are uninteresting, so try and exclude these from your report.
So for this task, you must submit:
(1) a single bash script, ’emowords.sh’ : as used to support your answers, perhaps calling ’emoword.py’; this should output for each of your 20 emoticons the most frequent words co-occurring with it (in tweets); use what ever format suits, as the results will be transferred and written up in your report.
(2) a description for this task is included in your final PDF report describing the method used for the bash script, and then the final list of selected interesting words per emoticon, and how you got them.
Task A.3 (2 marks)
See if there are other interesting information you can get about these emoticons. For instance is there anything about countries/cities and emoticons? Which emoticons have long or short messages? Whats sorts of messages are attached to different emoticons?
You can use the Python program ’emodata.py” which reads your ’emoticon.csv’ file, takes ‘msgraw_sample.txt’ as stdin and outputs selected data from the tweet file.
./emodata.py
Report on this in your final PDF report. Use any technique or coding you like to get this information. Your report should describe what you did and your results.
Task B
Consider the two files ‘training.csv’ and ‘test.csv’.
Task B.1 (2 marks)
Plot histograms of X1, X2, X3 and X4 in train.csv respectively and answer: which variable(s) is(are) most likely samples drawn from normal distributions?
Task B.2 (4 marks)
Fit two linear regression models using train.csv.
Model 1: Y~X1+X2+X3+X4
Model 2: Y~X2+X3+X4
Which model has higher Multiple R-squared value?
Task B3 (4 marks)
Now use the coefficients of Model 1 and 2 respectively to predict the Y values of test.csv, then calculate the Mean Squared Errors (MSE) between the predictions and the true values. Which model has smaller MSE? Which model is better? More complex models always have higher R square but are they always better?
3. Assessment Criteria
The work required to prepare data, explore data and explain your findings should be all your own. If you use resources elsewhere, make sure that you acknowledge all of them in your PDF report. You may need to review the FIT citation styletutorial to make yourself familiar with appropriate citing and referencing for this assessment. Also, review the demystifying citing and referencingfor help.
The following outlines the criteria which you will be assessed against.
3.1 Grading Rubric
The following outlines the criteria which you will be assessed against:
The marks are allocated as follows:
3.2 Penalties
4. How to Submit
Once you have completed your work, take the following steps to submit your work.
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"
2 Of 21 Scats Is A Computer Based Area Traffic Management System It Is A Complete Sy 2924876
/in Uncategorized /by developer2 of 21
SCATS is a computer based area traffic management system. It is a complete system that
includes hardware, software, and a unique control philosophy. The system operates in realtime, adjusting signal timings in response to variations in traffic demand and system capacity.
The purpose of SCATS, as with any area traffic control system, is to control traffic on an area
basis rather than on an individual, uncoordinated intersection basis.
New Generation SCATS
SCATS has been constantly improved and enhanced on a regular basis as new technologies
become available. The SCATS team has responded to the needs of the end-user and has now
released SCATS 6 which provides far more flexibility for the decision maker, the Traffic
Engineer and most importantly the accountant!.
This new generation has moved to a PC platform, has increased the number of intersections
that can be connected to one PC, has improved data collection resources and reporting
facilities along with improved management and monitoring methods.
SCATS 6 can be made available in the following formats with pricing to suit the end-user
based on their needs and budget.
Full Real Time Traffic Adaptive
Fixed Time Plans
Dial In Dial Out
Comparisons are available on request between the old and the new with the improvements too
large to itemise in this brochure. Details are available from the contacts listed on page 12, see
the hyperlink e-mail connection.
SCATS has always been a real time adaptive traffic management system. Nothing has
changed except the recognition of the variety of systems required by traffic engineers in
diverse traffic conditions around the world.
In this brochure SCATS is described in its most functional role, that of a real time, responsive,
adaptive traffic management system. Details of the method of fixed time plan operation are
available on request as well as the Dial In Dial Out system which offers unique remote access
to sites in outlying cities that need to be monitored on a daily basis reducing the need for
constant visits to check operation.
3 of 21
Adaptive System Copes with Unusual Demand
Adaptive SCATS, unlike “fixed-time” or “semi-responsive” systems, requires no precalculation of composite signal timing plans. Logic and algorithms in the system’s controllers
and traffic control computer analyse real-time traffic data from vehicle detectors to produce
signal timings which are suitable for the prevailing traffic conditions.
Many other area traffic control systems control the signals on a “fixed-time” basis in which a
series of signal timing plans are brought into operation at certain times of the day. Each plan
determines the timing of individual signals, and the time relationship between signals is precalculated based on previously surveyed traffic conditions.
A “fixed-time” system is generally unable to cope with unpredicted traffic conditions.
SCATS has addressed this problem when releasing SCATS 6 and has improved decision
making capabilities built into the FTP system to compensate for this deficiency in fixed time
operation.
No Need To Update Timing Plans
Furthermore, as traffic conditions change with the passage of time, fixed time plans become
outdated. This requires the area to be resurveyed and new signal timing plans calculated every
few years. Experience has shown this procedure to be expensive and to require resources
which are not always readily available.
As a result, the development of new plans is either deferred beyond the useful life of the old
plans or “ad hoc” changes are made to the plans and timetables, usually resulting in suboptimum performance.
More Responsive Control Method Required
The problems of “fixed-time” systems suggest that a technique more responsive to changing
traffic conditions would be more appropriate and more acceptable to the motoring public.
SCATS Offers Real Time Responsiveness
The implementation of a fully responsive system does not, however, mean that the careful
design of each intersection can be avoided. The present state of technology only allows for
the real-time variation of signal timings at the intersections which have been designed to suit
known or anticipated traffic requirements.
A degree of adaptability of the local design to varying traffic requirements can be
accommodated by a system such as SCATS. This is done by providing a variable sequence of
phases and the ability to omit phases or movements from the sequence on a cycle-by-cycle
basis, when there is no demand.
4 of 21
Four Modes of Operation
SCATS provides for operation of signals in the system under four modes:
Masterlink
Flexilink,
Isolated,
Flashing Yellow.
The adaptive mode, known as Masterlink, provides the integrated traffic responsive
operation.
In the event of failure of a regional computer or loss of communications, the local controllers
can revert to a form of time based coordination known as Flexilink. In this mode, adjacent
signals are synchronised by reference to the power mains frequency or an accurate crystal
controlled clock and signal timing plans are selected by time of day. Local vehicle actuation
facilities are operational in this mode.
Signals may also operate in an Isolated mode with local vehicle actuation being the sole
operating strategy.
The fourth mode is Flashing Yellow in which the normal signal display is replaced by
flashing yellow displays on all approaches or flashing yellow and flashing red to different
approaches.
Any Mode Can be Used
Provided communications are functional, signal operation can still be centrally monitored in
Flexilink, Isolated and Flashing modes. Any signal may be set to any of the four modes by an
operator using a SCATS workstation or by time of day.
Two Levels of Control
SCATS control of traffic is effected at two levels which determine the three principle signal
timing parameters of traffic signal coordination; phase split, cycle length and offset. These
two levels are referred to as “strategic” and “tactical”.
Strategic Control
SCATS “strategic” control refers to the top level of control which is impressed on a network
of coordinated signals by the regional computer. Using flow and occupancy data collected
from loop detectors in the road by the local controllers, the strategic algorithms determine, on
an area basis, the optimum cycle length, phase splits and offsets to suit the prevailing average
traffic conditions. This is carried out for adjacent groups of signals (usually one to ten in size)
which are known as subsystems.
5 of 21
Subsystems
The subsystem in SCATS is the basic unit of strategic control. Each subsystem consists of
one or more intersections and contains only one critical intersection which requires accurate
and variable phase splits. The intersections in a subsystem form a discrete group which are
always coordinated together and share a common cycle length and inter-related split and offset
selection. Phase splits for minor intersections in the subsystem are, by definition, non critical
and are therefore either non-variable or selected by a matching process which selects splits
which are compatible with the splits in operation at the critical intersection.
Subgroup Linking – building large coordinated systems automatically
To give coordination over larger groups of signals, subsystems can link together to form larger
systems, operating on a common cycle length. These links, which determine the offsets
between the subsystems, may be permanent or may link and un-link. This ensures that where
traffic flow between subsystems is sufficient to warrant coordination the link is enforced but
when one or more subsystems can operate more efficiently at a lower cycle time, the link is
broken.
Degree of Saturation
The basic traffic measurement used by SCATS for strategic control is the degree of saturation
on each approach or, more accurately, a measure analogous to degree of saturation. Inductive
loop vehicle detectors placed in important approach lanes at the stop line of the critical
intersections (and some detectors at other intersections) are defined in the regional computer
data base as strategic detectors. The local controller collects flow and occupancy data during
the green of the approach and, after pre-processing, it is sent to the regional computer and
used (together with automatically self calibrated saturation flow data for each detector) to
calculate the SCATS “degree of saturation” (DS).
DS is defined as the ratio of the effectively used green time to the total available green time on
the approach. The effectively used green time is the length of green which would be just
sufficient to pass the same platoon of vehicles had they been travelling at optimum headways
as in saturation flow conditions. The algorithm is capable of producing values of DS greater
than unity in congested conditions, enabling SCATS to deal effectively with over saturated
traffic.
Effect on Cycle Time
Cycle time is increased or decreased to maintain the degree of saturation around 0.9 (user
definable) on the lane with the greatest degree of saturation. A lower limit for cycle time
(usually 30 to 40 seconds) and an upper limit (usually 100 to 150 seconds) are specified by the
user. Cycle time can vary by up to 21 seconds each cycle but this limit is substantially
reduced unless a strong trend is recognised.
6 of 21
Effect on Phase Splits
Phase splits are varied by a few percent each cycle in such a way as to maintain equal degrees
of saturation on competing approaches, thus minimising delay. The minimum split which can
be allocated to a phase is either a user definable minimum or, more usually, a value
determined from the local controller’s minimum phase length. The current cycle time and the
minimum requirements of the other phases limit the maximum split, which can be allocated to
a phase.
Offsets
Offsets are selected for each subsystem (ie., the offsets between intersections within the
subsystem) and between subsystems which are linked together on the basis of traffic flow. In
this way, the best offsets are selected for the high flow movements. Other links carrying
lower flows may not receive good coordination if the cycle time is inappropriate. However,
when traffic conditions permit the use of a cycle time which can provide good offsets on a
majority of links, the system tends to maintain this cycle time even though a smaller cycle
time would provide sufficient capacity. Optimal offsets on the heavy flow links minimise the
total number of stops in the system, reducing fuel consumption and increasing capacity of the
system.
Tactical Control
SCATS “tactical” control refers to the lower level of control which is undertaken by the local
controllers at each intersection. Tactical control operates under the strategic umbrella
provided by the regional computer but provides local flexibility to meet the cyclic variation in
demand at each intersection. Tactics essentially provide for green phases to be terminated
early when the demand for the phase is less than the average demand and for phases to be
omitted entirely from the sequence if there is no demand. Conditional signal group
introduction is also provided. The local controller bases its tactical decisions on information
from the vehicle detector loops at the intersection, some of which may also be strategic
detectors.
Tactical Control is the Responsibility of the Controller
The tactical level of control is carried out in the local controller using exactly the same
operational techniques as described for isolated operation for a local controller. The degree to
which tactical control is able to modify the signal operation is entirely under the control of the
regional computer.
Tactical Control different to Isolated
A basic difference from isolated operation is that one phase, usually the main road phase,
cannot skip and cannot terminate early by action of gap and waste timers. This is because all
controllers in a linked group must share a common cycle time to achieve coordination. Any
time saved during the cycle as a result of other phases terminating early or being skipped may
be used by subsequent phases or is added on to the main phase to maintain each local
controller at the system cycle length.
7 of 21
Strategic and Tactical Control Together Equals Efficiency On The Street
The combination of strategic control which varies the split, cycle time and offsets in response
to gradual changes in traffic demand patterns together with tactical control which handles the
rapid but smaller changes in demand cycle by cycle results in a very efficient operation of the
signals on the street.
Operator Control
SCATS provides the operator with a range of manual functions to override the normal
automatic operation. These functions allow manual control of signal lamps to “on”, “flash” or
“off”; manual selection of link mode to Masterlink, Flexilink or Isolated mode; manual
selection or alteration of split, cycle time or offset either on an individual intersection or for a
whole subsystem; a dwell facility which allows any signal to be held on a nominated green
phase for as long as required.
Variation by Timetable
SCATS also allows for system operation to be varied by a timetable. Almost any function
which can be executed manually can also be set up to occur at specified times on specified
days. For example, in a central business district, pedestrian walks may be automatically
introduced on business days, late shopping nights and other periods of high pedestrian
activity.
Special Routines
A range of special routines is also available in SCATS which allows the user to define special
operations to occur under special conditions. These routines are used to address requirements
not covered by the general operation of SCATS. It is features of this type which enable every
detail of signal operation to be tailored to meet the operational needs of each individual
intersection. SCATS is the only system to offer such a feature.
Capacity
The theoretical capacity of SCATS regional traffic control software is 250 intersections.
Software is available in a variety of increments as follows:
0-16 Intersections
17-32 Intersections
Subsequent increases are in multiples of 32 Intersections
FALLBACK OPERATION
Automatic Fallback
In the event of regional computer failure, loss of communications between the computer and
any local controller, failure of all strategic detectors or certain other local malfunctions, the
affected intersection(s) will “fallback” to a user defined mode of operation which may be
either Flexilink (time based coordination) or Isolated operation.
8 of 21
Coordination Maintained During Fallback
If specified by the user, fallback at one intersection will also cause other intersections in the
subsystem to fall back and, optionally, intersections in adjacent linked subsystems. In this
way, if Flexilink is specified as the fallback mode, coordination can be maintained between
intersections affected by the failure.
The Controller is the Key
All data necessary for fallback operation is held in the local controller, ie., local signal timings
for Isolated operation and plans and schedules for Flexilink operation. A copy of this data is
held in the regional computer so that it may be “downloaded” from the regional computer to
the local controller in the event of it being lost. The clocks in the local controllers are
periodically checked by the regional computer and adjusted as necessary.
SYSTEM HARDWARE – COMPUTER PLATFORM
Distributed, Hierarchical System
SCATS has been designed in a modular configuration to suit the varying needs of small,
medium, and large cities. Personal Computers are used. In its simplest form, a single regional
computer can control signals at up to 250 intersections. Expansion of the system is achieved
by installing additional regional computers. For large systems, it is usual to add a Central
Management Computer which provides centralised access for data input, monitoring and
traffic data collection, improved system management support, data analysis, data backup, fault
logging and analysis and a system inventory. These features ease the logistic burden of
managing larger systems. A typical large system SCATS computer configuration is shown in
Figure 1.
Regional Computers
Personal Computers operating under the Windows NT operating system are used for the
SCATS regional traffic control function. This software operates on standard PCs and The
Digital Alpha range of computers. Asynchronous serial (multiport) interfaces and modems
(one channel per intersection controller) interface the Regional Computer to the intersection
communications lines. Regional computers are usually located near the centre of the group of
signals to be controlled in order to minimise the cost of communications lines.
Central Management Computers
SCATS Central Management Computer can be a Personal Computer operating under the
Windows NT operating system. Communications with Regional Computers and
Workstations is via an Ethernet LAN or, for serial communications with remote Regional
Computers, via a PC based Communications Processor.
9 of 21
User Interfaces
Operator interface to SCATS is normally by a Personal Computer (PC) acting as a
workstation terminal and running RTA interface software. Minimum requirement is 486 DX
or better with a 32 bit operating system (Windows 95 or Windows NT V4). The graphical
user interface requires a minimum screen resolution of 800 x 600 (Super VGA).
PC workstations are able to operate in the following modes:
Local mode (as a free standing PC)
Local network mode (accessing any computer on the LAN)
SCATS workstation, providing access to the traffic control system and management
subsystems provided by the Central Management Computer.
Workstations may be connected via the LAN (eg thin wire Ethernet), via a terminal server
or direct to a Regional Computer.
Field terminals (eg laptop PC) connected to a local controller are also supported.
MONITORING AND CONTROL FACILITIES
The full range of operator commands and monitoring functions is available from all
workstations subject to the security access afforded to each operator as defined in the
database. Passwords are provided for security purposes. These facilities are provided from
workstations at the control centre, any regional computer, and any intersection controller or
remotely via modem. The data displayed includes:
For Intersections:
For Subsystems:
Lamps ON/OFF/FLASHING
Current phase demands
Detectors occupied
Cycle length
Operational mode
Alarms
Phase running
Time in Phase.
Current splits
Current offset plan
System cycle length
System detector data
Graphical User Interface
A graphical user interface (GUI) is now the standard user interface, replacing the previous
character based screen. The intersection monitoring window with a small intersection graphic
display is illustrated in Figure 2. Data entry is by forms, an example of which is shown in
Figure 3. All alarms are logged and can be viewed with the Alarm management window as
seen in Figure 4.
10 of 21
Graphics
The PC workstations support full colour graphics in a sizeable window. Four levels of colour
graphics display are provided:
The Server window: Figure 5 shows a map of the whole system indicating by colour the
boundaries of each region and the traffic conditions at the six most heavily trafficked
subsystems in each region.
The Graphics window Regional display: Figure 6 shows a map of the selected regional area
with an on-line representation of traffic flow conditions by means of colouring the roads
with five different colours representing traffic conditions in the range from very light traffic
to heavy congested conditions.
The Graphics window Subsystem display: Figure 7 shows the selected subsystem layout
together with an on-line graphical bar chart representation of traffic flow and density as
measured by the strategic detectors in the subsystem.
The Graphics window Intersection display: Figure 8 shows the selected intersection layout
and phasing design with real time display of detector operation and phase greens.
On-Line Control
It is possible to display and/or change all adaptive control parameters from any system
workstation while the regional computer is on-line both by operator command and
automatically by time of day. There is no need to take the regional computer off-line when
altering data or re-configuring the dimensions of any data array. Manual control of any
intersection is also possible from any system workstation.
Alarm Conditions
The system provides a comprehensive set of alarm conditions to warn the operator of all
unusual or fault conditions. These alarms are logged automatically on occurrence and
clearance and can be queried at any time. Alarms are also provided for congested traffic
conditions in each subsystem.
SYSTEM HARDWARE – LOCAL CONTROLLERS
Four Controller Modes
SCATS local controllers can operate in four modes. These modes can be invoked manually or
automatically by the regional computer or at the local controller.
11 of 21
Masterlink Mode
In the adaptive mode the regional computer determines the phase sequence and the maximum
duration of each state and the duration of walk displays. The local controller may terminate
any phase under the control of the local vehicle actuation timers or skip an undemanded phase
unless prohibited by instructions from the regional computer.
The regional computer controls the phase transition points in the local controller subject to the
local controller safety interval times being satisfied (eg. minimum green, pedestrian clearance
etc.). On completion of the transition to a new phase, the local controller times the minimum
green, and minimum walk intervals and then waits for a phase termination command from the
regional computer. On receipt of the command to move to the next phase, the local controller
then independently times the necessary clearance intervals (eg. yellow, all-red) for the phase
termination.
Communications errors or faulty operation of the traffic control computer cannot cause the
local controller to produce dangerous signal displays such as short greens, short pedestrian
clearances, short yellows or short reds as would be the case if the local controller depended on
the regional computer for the timing of all intervals.
The termination of pedestrian walk signals is also under the control of the regional computer
so as to allow the walk timing to be varied to match prevailing traffic conditions. The
duration of the walk signal cannot, however, be less than the prescribed minimum walk.
Flexilink Mode
In Flexilink (time based coordination) mode the phase sequence and the maximum duration of
each phase and the duration of walk signal displays is determined by the current plan. The
local controller may terminate any phase under the control of the local vehicle actuation
timers (gap, headway and waste) or skip an undemanded phase unless prohibited by
instruction within the plan. Flexilink is the usual fallback mode of operation.
Isolated Mode
In Isolated mode the state sequence and the maximum duration of each phase is as specified in
the local controller time-settings. The local controller may terminate any phase under the
control of the local vehicle actuation timers (gap, headway and waste) or skip an undemanded
phase unless prohibited by the local controller personality. Isolated mode may be specified as
the fallback mode of operation.
Flash Mode
In Flash mode the signals display flashing yellow to all approaches. Other flashing displays
can be provided eg., flashing red/yellow.
12 of 21
Phase Sequencing
The signal cycle is divided into phases called A, B, C, etc., and these can be introduced in any
defined sequence (eg., A-B-C-A). Any phase can be skipped if no vehicle is waiting for a
green on that phase (eg., if no vehicle is waiting for B phase the sequence would be A-C-A).
In Isolated and Flexilink modes, the sequence is as defined in the local controller personality.
In Masterlink mode, the sequence is determined by the regional computer.
DETECTION
Stop Line Detection
All detectors (both strategic and tactical) are normally located at or near the stop line (one in
each lane). The calculation of DS relies on the detector being of sufficient length in the
direction of traffic flow to ensure that large values of space are not measured under conditions
of slow moving, closely spaced traffic (which would appear to be the same as light traffic
widely spaced). The detector must not, however, be too long as it would not measure any
spaces when traffic moves freely. Research has shown the optimum length of the detection
zone to be 4.5 metres.
Strategic Detectors
Strategic detectors are located at the stop line in order to enable measurement of the use made
of the green time by traffic at a point at which the traffic is controlled by the signal. If the
strategic detectors were placed remotely from the stop line, assumptions would have to be
made about the flow rate actually achieved during the green period. At any time when these
assumptions were not valid, an incorrect green time would be allocated to the approach.
Tactical Detectors
Tactical detectors located at the stop line enable differentiation between the left turn, straight
ahead and right turn movements at the intersection both by knowledge of the lane usage in
lanes of exclusive use and by speed differential in lane shared by two or more movements. If
the detectors were remote from the stop line it would not be possible to identify the intended
movement (direction) of detected vehicles due to subsequent lane changing. Additional
detectors may be installed in advance of the stop line but this has, in general, been found
unnecessary.
Detector Requirements
Tactical detectors should be provided on all lanes of an approach (or movement) which will
benefit from tactical control, the more minor movements being the most suitable.
It can be seen that approaches most requiring strategic detection are those least requiring
tactical detection and vice-versa, resulting in the need for detection on most approaches. In
general, the approach lanes which can be left undetected are lightly used curb lanes on
approaches which otherwise require strategic detection and at minor intersections on the
“main road” approaches which are not immediately upstream of a major intersection.
13 of 21
COMMUNICATIONS
Two Wire Circuits
The local controllers are connected to SCATS by standard voice grade telephone lines or a
dedicated cable network. In both point to point and multi-drop configurations, a single pair of
wires is required.
Communications Mode
Messages are sent to, and a reply message received from, each intersection controller, every
second.
In point to point mode data is transmitted at 300 bps full duplex, asynchronous, FSK.
The low speed rate required for SCATS communications allows for a high degree of tolerance
in the reliability of the local communications network.
Roads and Traffic Authority of NSW Contact
Traffic Systems Branch
RTA Transport Management Centre
25 Garden St Eveleigh NSW 1430
AUSTRALIA
Postal Address:
Traffic Systems Branch
RTA Traffic & Transport Directorate
PO Box 1927
Strawberry Hills NSW 2012
Australia
www.rta.nsw.gov.au
Attention: Jim Giffin
Phone: +612 8396 1605
Mobile: + 61412 251 860
Fax: + 612 8396 1600
E-mail: [email protected]
SCATS Distributor Worldwide Contact
Tyco Electronics Products Group
Unit 1
2-8 South Street
Rydalmere NSW 2116
Australia
Attention: Allen Yip
Phone: +612 9638 8212
Fax: +612 9638 8113
E-mail: [email protected]
14 of 21
Figure 1
Central
Management
Computer
with CMS
Workstations
Communications
Processor
Regional Computers
(serial communications)
Regional Computers
(LAN connected)
Typical SCATS Computer Configuration
15 of 21
Figure 2
Monitoring window with small graphic window
16 of 21
Figure 3
Data entry form
17 of 21
Figure 4
Alarm Manager
18 of 21
Figure 5
Server Window
19 of 21
Figure 6
Region display
20 of 21
Figure 7
Subsystem display
21 of 21
Figure 8
Intersection display
"Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!"