software development in agricultural computer science ... · the cathedral & the bazaar (eric...
TRANSCRIPT
Software Development inAgricultural Computer Science:
Status and PerspectivesEildert Groeneveld
Rodica, April 15th. 2004
Institute for Animal Breeding, FAL, Mariensee
Software Development in Agricultural Computer Science: Status and Perspectives – p. 1
Outline
1. Objectives in Computer Science in Agriculture
2. The Framework
3. Thesis
4. the “so what?” projects
5. examples of dead-end developments
6. possible solutions and development paths
7. examples of the OS development model
8. Open Source development
9. outlook
Software Development in Agricultural Computer Science: Status and Perspectives – p. 2
Objectives in Agr. Computer Science
Facilitate IT solutions in Agriculture
quick development and implementation of new solutions
requires research and development
essentiell framework:
cumulative mode of developmentleads to accelerated developmentis based on original research
progress resulting from the general development inSW/HW does not count
Software Development in Agricultural Computer Science: Status and Perspectives – p. 3
The Framework
Information processing in Agriculture:
in the scientific area:
design and evaluation of experimentsdevelopment of computing algorithmswell defined inputs -> problem solution -> Output(IPO)
Software Development in Agricultural Computer Science: Status and Perspectives – p. 4
The Framework ff
Information processing in Agriculture:
on farm area:
solutions to local problems: optimization of feedrations (IPO)area of global information: collection, managementand evaluationmerging data from various sources:
lots of land, soil samplesperformance recording: milk, growth, reproduction
across farms:
aggregation of data from many sources (InfoSys)in animal production: BLUP genetic evaluation
Software Development in Agricultural Computer Science: Status and Perspectives – p. 5
The Framework fff
Information processing in Agriculture:
animal breeding, crop production, agriculturalengineering, agricultural economics
different kind of systems (IPO ⇔InfoSys):
IPO: production of data -> problemsolution->decision->throw data away
InfoSys: integrated data utilization: breeding, milkrecording, labs, AI, utilization of the WEB,networking of farms
Software Development in Agricultural Computer Science: Status and Perspectives – p. 6
IPO vs InfoSys
conceptual differences:
IPO:
simple well defined inputscomplex processing but usually in a simple programenvironment (F90, C, ...)well defined outputsprocessing layer is disconnected from the inputs
Software Development in Agricultural Computer Science: Status and Perspectives – p. 7
Examples IPO
statistical procedures through use of packages (SAS,R...)
analysis of inbreeding (SW)
computation of BLUP (SW)
estimation of covariance components (SW)
etc.
Software Development in Agricultural Computer Science: Status and Perspectives – p. 8
IPO vs InfoSys ff
conceptual differences:
InfoSys:
many sources of data from different origin/systemsas a result: different SWas a result: different operating systemsincludes data transferdifferent data structures among different problems(organizations)everything is connected with everything
Software Development in Agricultural Computer Science: Status and Perspectives – p. 9
Examples InfoSys
gene banks in plants and animals
e-Commerce Systems
query and information systems
integrierted data bases: plants/animals
many components:
Webserver,DBMS,programming languages,OS
Software Development in Agricultural Computer Science: Status and Perspectives – p. 10
Thesis
IPO:
1. many developments with large impact: DFREML,MTDFREML, ASREML, VCE, ABTK, PEST
InfoSys:
1. many developments that solve ONE problem
2. are thus irrelevant for the industry as a whole:
3. the “So What?” projects.
Software Development in Agricultural Computer Science: Status and Perspectives – p. 11
The Problem: An Example
WCGALP Australia in Section :”Computing Techniques in Animal Breeding”
total of 26 contributions
17 IPO
4 InfoSys:
NZ sheep improvement schemeQTL detectionWEB Animal Recording SchemeAMNOS Dairy Sheep Managementall SW
Software Development in Agricultural Computer Science: Status and Perspectives – p. 12
Analysis of the Problem: own examples
1970: central database for the BHZP:
HP-1000network type databaseelectronic on-farm data recordingelectronic automated data transferremote program execution
Effect:
solves the BHZP problem, but no other despitepublicationsorry to say: SW project
Software Development in Agricultural Computer Science: Status and Perspectives – p. 13
Analysis of the Problem: own examples ff
End 1980:
new attempt at generalization:ORACLESUN-OSpig breding program Sloveniatransfered to Chinese Swine Herd at U of I (USA):no transfer beyondsolved a problem in Slovenia and U of I but nothingelse despite Master’s thesissorry to say: SW project
Software Development in Agricultural Computer Science: Status and Perspectives – p. 14
A Key Experience
meeting of developers of integrated databases
different countriesdifferent external circumstancesremarkable:
different systems and tools (dBase, Clipper,Oracle...)totally different development pathsevery system a completely new developmentwithout reference to previous work (literature?)no way to use software acrossnothing worked really well
Software Development in Agricultural Computer Science: Status and Perspectives – p. 15
Critical Question
How manydevelopment projectsin the different areas
are of the kind:“So What”?
Software Development in Agricultural Computer Science: Status and Perspectives – p. 16
What to do?
the basic problem:
solution of ONE problemthus: not transferablebecause solution of one problems includes all sideconditions:
operating systemRDBMSsystem tools and software
this is hardly ever an issue in ONE problembut strong restriction on transfersoftware is generally not available
Software Development in Agricultural Computer Science: Status and Perspectives – p. 17
What to do? ff
solution can only lie in:
generalization: find the Poodle’s corealso: never develop for ONE case, but always formanyparallel implementationimplement a layered system which allows replacingcomponents (non monolithic)release software
Software Development in Agricultural Computer Science: Status and Perspectives – p. 18
A New Development Path
started as “Integrated Database of PerformanceRecords in Pig Populations”
any animal identification systemany data structureinstallation free of licencing costsscales from on-farm to nationaldevelopment of procedures for development andmigrationdevelopment of software toolboxdevelopment of general evaluation software
Software Development in Agricultural Computer Science: Status and Perspectives – p. 19
A New Development Path ff
avoid proprietary software
use standards (SQL-99)
use Open source
developer group
implementation in at least two environments
Software Development in Agricultural Computer Science: Status and Perspectives – p. 20
Development
Strategy/Procedure
can be used independently from software⇒migration steps of an information system
Software
for the implementation of the procedures
generic solution for BR
Software Development in Agricultural Computer Science: Status and Perspectives – p. 21
Closed or Open Development?
IPO can work without program modification
InfoSys: different data structures require softwaremodification⇒ System without source code not conceivableclosed development:
few people have the knowledgerestricted pool for future development
need to create development capacity⇒OpenSource
Software Development in Agricultural Computer Science: Status and Perspectives – p. 22
Open Source Development
The Cathedral & the Bazaar(Eric S. Raymond)
“given enough eyeballs, all bugs are shallow”
big project:
EMACS, LINUX, FETCHMAILKDE, GNOME
requires infrastructure:
Internet connectionSoftware management system (CVS)
Software Development in Agricultural Computer Science: Status and Perspectives – p. 23
Open Source: Advantages
get help with debugging
expand developers base
enhances future system safety through expandedknowledge base
help during conceptual development
cost effective developer and test capacity via theInternet
the world is the limit (cooperation space)
Software Development in Agricultural Computer Science: Status and Perspectives – p. 24
Implementers
Institute of Animal Production, Irene, South Africa
Landesanstalt für Tierzucht, Köllitsch
University of Ljubljana, Slovenia
Research Institute Nitra, Slovakia
University Debrecen, Hungary
Veterinary Academy Kaunas, Lithuania
University Stara Zagora, Bulgaria
FAO
University Göttingen
TZ, Mariensee
Software Development in Agricultural Computer Science: Status and Perspectives – p. 25
Current Projects
PISSA South Africa, Pigs, Cheetah
Beef South Africa, Cattle
PIS Lithuania, Pigs
CryoDB Netherlands, Germany, Genebank
MINIPIGS Göttingen, pigs
MOLABIS Germany, DNA Material
SCHAF-OT Germany, Sheep
Saxonian Pig System, Germany
EFABIS (FAO, F, PL, D), Biodiversity Database
a number in Slovenia: Rabbits, Horses, Cattle, Pigs, ..
Software Development in Agricultural Computer Science: Status and Perspectives – p. 26
Platforms create Utility
many developers in CC, Research Inst. but SWgenerally not usable in InfoSys
on generic platforms software can be used:
Population reportZWISSS...
cumulative development model
Software Development in Agricultural Computer Science: Status and Perspectives – p. 27
Population Report 1
�� � � �
��� �� � � � � � ��� � � � � � � � � � � � �
��� ��� �� � � ���� ��� �
� � � � �� � � ! � � "#� ��$ % & � $ ' � �( � $ � � �
") *+ , %- ./ � � � � � 0 01 /
2354 6 %� % � / 0 03
78 9 :; < < =
>@? A ? B ?C 9
Software Development in Agricultural Computer Science: Status and Perspectives – p. 28
Population Report 1 ff
� �� � � �� � �� � �� � � � � � � �� � � � � � ��� � ��� � �� � �� � � � �
� "! #! $ % &
' (*) + , ! $ - .*/ 0 $! / % # 1 1 % +/ 0 # $! 2 $ - 1) 3 4 0 - # ,*5 5 ! % $
� 67 � � �8 9 � � : 7 � � � � 6� � � ; �� � 9 < � 9 �� � �� � � 9= � � �� > 7 � � �8 � 9 > � ? � 7 9 � � @�
6 �A � 9 � � : 7 � � 8 � �B � �� C �� � � � � :� � D � � 6� E � � �8 �� � 9 < � 6� � � � :� : 7 � � � � 6� � 7 �� �
< 9 : : 9= � > �B � 9= � � � A � �B � �F � 7 9 � 6 �� � 6 �� � F 9 : � ; � � � � 6� B �� � 8 ��� � � � C � 9 �� �
� � > � 9= � DHG
� � � � � > 7 � �� � A 7F � G 6� �� = � F 9 � � � � 6� � � ; �� � 9 < � 9= � � � > � 9 �� � = 6 98 �� �� 7F 78 � �� � 7 � �� � A 7F � � > � � 7 �I � 6� F 9 � � � �8 9 � > 7 � I B � �� �
� � 7 � :7 � �� � � G � 6 7 � 7 � � 6� � 9 � � : � � ; �� � 9 < � 9= � �� > � 9 �� � = 7 � 6 : 7 � �� � � 7 � � 6�
I 7 A � � B � �� �
��� I 7 A � � � 6� � � ; �� � 9 < � 9 �� � �� > � 9= � 6 �A 7 � I �� :� F �� > 9 J�8 � 7 �I 7 � � 6�
I 7 A � � B � �� � �� :�F � 7 9 � 7 � >� E � � > �� 6 �A 7 �I � E � � � �� � A 7F � � � 6 � �*K � 67 � 7 �
� 6� � � ; �� � 9 < � 9= � �� > � 9 �� � = 7 � 6 � E � � � �� � A 7F � 7 � � 6� I 7 A � � B � �� �
� � �� : :� �G � ; �� � 9 < � 7 �� � �� > > � ;� 7 � ��8 � 9 > �F � 7 9 � �B B � �� C � @ D
� � �� � � ; �� � 9 < � 7 � � � � � ; �� � 9 < > � ; �
�� � A 7F � � : 7 � �� � � 9 J�8 � � �� :� � � � A 7F � � : 7 � �� � � 9 J�8 � � � � :�� L M L � � � � � �
� LN � � � � �
� LN � O � � O � �
� LN O � O � � O �
� LNP O Q � Q M �
� LN Q N O P � � M Q
� LN M O M O � � � � P
� LN N � � P Q � O � � � �
� LN L � � � � N � M � O � O
� L L � � O � � � � � L � Q � O
� L L � � � � L � � ON O Q � Q
� L L � � Q � L � N P N ON � O
� L L � � � � � � Q M � Q O � N
� L L O � � � � � � MN M Q �P
� L LP � N � N �P M M N � � �
� L L Q � Q �P � � P � P � � L
� L L M � � � O L � � O � � �
� L LN � M � � N
Software Development in Agricultural Computer Science: Status and Perspectives – p. 29
Population Report 2
�� �� �� ��
��� � �� � � � � �� � �� �� � � �
�� ���� � � � � �� �� �� ��
� ! � � �" �� # � � $ ���% & � ')( �� �(� & �� $ ���% & �* " ! + & � � �,
-./ -/ 0� " ! � &� � 1 & ��� � !� �
243 5�" ��6 7 7 -
89 :9 ;9 <= > <?@ @ ACBD EB
F
Software Development in Agricultural Computer Science: Status and Perspectives – p. 30
Population Report 2 ff
� � �� � �� � �� � �� � � � � � �� � ��� � �� �� � ��� �� � � �� ��
� ! � � �� "# � $ ! �% & '( �% � ( � ) ) $( �� % �* % & )� +� � &� !, , � % - � & .
� � % � � $ ! �% & ' ( �% � ( � � $ ! �% & ' ) $(
( �% / � +� ( � �� � �% ( & 0( * % � ( � � � ( �% / � +� ( � �� � �% ( & 0( * % � ( � � �"1 2 3 41 � 2 5 2 " 2 " 1 4 " � 3
"1 2 5 2� 4 � 5 2 " 2 3 "� � 1 "
"1 2 4 � " 51 4 4 �� 5 " � � 2 4
"1 2 � � " 51 5 � " 4 " "� " 41
"1 2 2 4� 4� 5 4 " 5� "� � 2�
"1 21 4 � 5� 5� " �1 " � 4 � �
"1 1 � 31 � 1 5� " � � � " 4�
"1 1 " 4 � 5 2 5 " " � � 2 4 4 2
"1 1 � 4 � 4� 5 3 " 3 5 2 3 4 4
"1 1 � 3 � 3 4 31 2 4 41 5 5
"1 1 3 " " " " � " 5 " 4 �
"1 1 5 3 " � � 4� 41 � � 2 �
"1 1 4 2 4 5 � � 3 5
"1 1 � " 5 " � " 2 3� � � ��
"1 1 2 �1 � � � 2 4� � � � 3
"1 1 1 �� �� " 3 � � � 2 � "
�� � � � 3 " 3 � 5 � 4 �� � 2
�� � " �1 �� � � � 5 � 3 ��
�� � � " � �� " � �� � " "�
�
Software Development in Agricultural Computer Science: Status and Perspectives – p. 31
Publications
is SW development in InfoSys or IPO research?
I think it is. Need to change attitudes.
general problem with OS projects
may be: we are part of the problem?
Software Development in Agricultural Computer Science: Status and Perspectives – p. 32
Summing Up
Commercial Development: solve ONE problem
Research in CS: develop generic solutions
research in this area is not uncontested
clean definition of layers
Open Source
avoid proprietary solutions
push for a cumulative development model
Software Development in Agricultural Computer Science: Status and Perspectives – p. 33