Take-awaysFeatures of this article that apply to good research

Please notice the elements of a strong experimental design• In their results, they pick random values for parameters (such

as C,M,R,D) but then try them for a variety of values. They had to pick certain values. If they just chose a certain value, someone would criticize by saying – “Hey, that isn’t a good value.” By using a “radio button” so the values are easy to change, you eliminate that source of criticism by saying, “What value would you like to use? I can do that.” Or “We experimented with different combinations and found the results to be similar.”

• They run the tests 1000 times each and average the results. This is very important in showing the results are repeatable and statistically significant.

Take-awaysFeatures of this article that apply to good research

• Their tests are set up for comparison. We judge goodness by comparison to other methods. Their definition of optimal (central) serves a valuable benchmark. They also compare their method to itself with different parameters. This is a very easy test to make (rather than trying to compare to other researcher’s results – which involves implementing OTHER algorithms)

• When they show us results, they don’t leave it up for interpretation. They tell us what the charts mean – what it shows and why they got those results. This is great for two reasons. First, they know they communicated what they intended. Two, we have a sanity check to make sure we understood what was happening.

Take-aways

• Their tests isolate individual factors. How important was the “win or learn fast” idea? How important was the number of services each agent could do? How important was the similarity between tasks?

• They give us so many results, that we really don’t even notice that their method is only 15% better than random adaptation. That doesn’t seem very impressive.

• Results are so important in getting the paper published. Reviewers are trying to find a reason to eliminate your paper. Many times I’ve been reading a paper and thought, “This is a bad idea. This will never work.” But when I get to a well prepared results section, my attitude changes, “This is better than I thought. They have actually implemented their ideas so they understand the issues far better than I.” Without the results section, nothing changes my negative first impression.

Take-aways• There were a few “ordering” problems. When each agent

summarized its subordinates, it wasn’t clear whether or not capacity or current load was considered. Nor was it clear if agents really had multiple skills (as examples didn’t show it). Some answers appeared much later. Others were never clear. It would help if precise limitations of the model were presented initially. I would have liked statements such as – “While agent summarize their subordinates’ skills, this is far from perfect

as capacity is not recorded, availability cannot be tracked, and we cannot distinguish between one agent having four skills and four agent who each have one.”

– Similarly, I would have liked a clarification such as “Tasks are assigned based on availability of agents possessing the required root skill. In case of a tie, the determination is made randomly. No attempt is made to match a task to an agent having subordinates who can provide needed services.”

Self-Organizing Agent Organizations

Kota, Gibbins, Jennings

Hints for reading papers• Need to find an organized way of making progress reading

unfamiliar material• Need to find a way of isolating problem areas so they don’t

become a “show stopper” to your progress• Sometimes all definitions are there – just not in the right

order.• Sometimes you don’t know what you don’t know –

misunderstanding terminology, for example.• Sometimes finding other sources of material is needed (other

articles by same authors or even google)• Wikipedia is great for understanding basic terms!• Practice reading unfamiliar material and being able to

summarize the main points.• Blue means this is my opinion or something I’m unclear on.

Big Picture – Main Idea• You have tasks consisting of multiple kinds of

services that need to be done• You have agents that can do the work.• Without using a centralized manager, agents

need to decide who will work together• To avoid lots of searching/requesting for who

should be in the coalition, agents organize themselves using “links” or “relationships”. The links represent agents they prefer to work with.

Agents• In any coalition formation system, we have tasks and agents.

Somehow they must be matched.• Agents: have a set of services/skills they can perform (e.g. s3)

and a capacity (how much work they can do it a time step).• The capacity is consumed as agents engage in services.• It appears that there is one capacity (not a different capacity

for each different service). Agents being able to do multiple services definitely complicates the model.

• Agents also have connections to other agents (friendships). In this setup, there is no requirement that agents be related in a special way to do the task. The links are mainly for efficiency (so we don’t have to search all agents).– acquaintance– peer– superior/subordinateIt is the management of the links which is the focus of the paper.

What the links mean…• The agents know about the existence of every

other agent• Some agents are related by a

“superior/subordinate” relationship. The superior can push work to the subordinate who WILL do it if they can.

• Other links are called “peer” links. A node can also contact a peer about doing work.

• I don’t get a clear idea of when/if they can refuse work.

• A node who can’t get the work done can pass it back to the superior or fail.

Agents• Agents who are busy (but have the skill to do the required work)

can delay the task. Delayed tasks are worth less.• Agents can work on multiple tasks at the same time if they have

sufficient capacity.• While agents keep track of what their subordinates can do and

their peers, they don’t appear to keep track of other agents’ skills – BUT they may.

• Also, it doesn’t appear that the agents know the current capacity (for subordinates) for doing the tasks. That would be difficult to keep current. Key data structuring concerns: how access (by skill or by capacity)? how keep current?

• I would also think that if the agent kept track of how much total capacity their subordinates had that could be useful.

• If agents have multiple skills, the summary is not as helpful as it could be as it could look like you have four skills in your subordinate set, but really you have one agent with four skills.

Agents Reward• It doesn’t appear that tasks have rewards that

are different than the amount of work to complete them. There are no “deals”.

• It doesn’t appear that agents really receive individual awards – other than possibly getting credit for the time they were busy (less the time they spent doing overhead stuff).

• Agents just want the system to succeed.• Thus, the issues of deciding who gets what is

missing.

Tasks• Tasks are in a treelike structure where the links represent a “must follow”

(precedence) relationship.• Each node represents a (service/skill, capacity) pair that is required to

execute that part of the task• Tasks are assigned to a randomly selected agent (I’m assuming the agent

has the root skill and that it is currently free). Is any attempt made to match task to an agent subtree which is more capable?

• Tasks should start executing immediately• An agent is responsible finding an agent to execute all dependent tasks.• Thus, a task may look like:

Tasks• The reward associated with a task is the sum of the

capacities required. I’m guessing this is done for simplicity. The idea is that the worth of a task is proportional to the work required.

• Tasks also have an associated complete time.• Tasks that don’t complete on time degrade linearly. At

first glance it appears that you either go negative or break even – but they don’t pay the agents for completing the task. In essence the reward is the total amount of time agents are busy doing tasks that are completed LESS penalties paid for being slow.

• Tasks are not known by everyone.

Organization• For efficiency, agents keep track of the skills present in their

descendants. This diagram shows separate lists of skills in descendants.

• There are other (better?) ways of keeping track, but the idea is that the nodes have some summary info.

• Reorganization involves creating new links or dissolving old links.

• The number and organization of links affects overhead costs.

Organization• Agents try to do the tasks if they can. So there is

no autonomy in that regard.• Agents can pass task to subordinate, peer,

superior. We don’t know when it decides just to wait and do the task itself, later.

• On the subordinate/superior links, it isn’t clear what is allowed. It appears that we need to form a non-cyclic graph – but it doesn’t need to be treelike. This matters in the summary information IF you are keeping track of capacities. You might have the same descendant through two different paths.

Idea• The goal of the paper is to have the agents

change their links to facilitate better coalition formation.

• Adapts to changing conditions and environment.– adding/deleting agents (dynamic or open system as

opposed to a closed system)– Tasks with different skill combinations

• Not worried about trust, negotiation or coordination of services.

• Not smart enough to say, my superior has a task for me so I won’t do something for a peer.

Terms• centralized vs decentralized control.– Agents want to be autonomous– Centralized systems are not “agent-like”

• robust – failures don’t bring it down• self-organizing – agents use local information to

make decisions about structure (rather than relying on central control)

• working towards organizational goals (not totally self-interested)

• meta-reasoning – reasoning about reasoning. Used here to mean they reason about whether or not they want to consider adaptation

Model

• Tasks enter the system at various times• Costs are associated with passing messages,

doing the work, reorganizing.• Agents perform execution and allocation.

Both take part of their “capacity”. The work associated with allocation is dependent on the number of links it has to search.

Performance Evaluation• Profit: sum of value of completed tasks minus costs of

allocation reorganization. • I don’t see that the cost of the agent working is

counted – so if an agent is idle, is there no penalty? They do say an idle agent may as well be planning its reorganization – so how it is we don’t account for all resources consumed?

• Lots of notation – so highlight the terms so you can refer back to them.

• Assumes there is some cost per link looked at and some cost for each reorganization done. No values are given.

Costs

• We don’t appear to get any benefit for a partially completed task

• Agents don’t appear to be able to abandon tasks that don’t seem profitable

• Tasks essentially expire (in that they have been delayed too long) – but we don’t seem to consider the length of time a task has left in committing to it.

Reorganization

• Agents can change a link with anyone: a peer, subordinate, superior, or a mere acquaintance.

• Lots to consider, so decide to consider a small subset.

• One decision can be “no action”• Need to evaluate the utility of each change. Is

it possible that a pair of agents may disagree about the best link between them?

Reorg costs (between x and y)

1. change of load on x2. change of load on y3. change in load to others4. change in communication cost5. reorganization cost (Isn’t most of the cost

felt in deciding about reorg, instead of actually doing it?

Costs are jointly computed.

Value

• The change in load can be negative. I’m confused as to what this means.

• They say a negative load is an increase in load which represents a decrease in utility.

• If I have less load, that means I have less to do – so I can’t contribute?

• However, if I have too much load, I don’t get it done – and that doesn’t help anyone.

• What are they measuring?

Load• I don’t think we want to interpret load too casually. It is a value

associated with workload, but we need to understand its range and meaning.

• Consider the load for adding a new subordinate• Δload = -M*SIs allocated * percent can’t satisfy load• M the management load to consider a subordinate• the result represents the additional cost that would have occurred

if relationship under consideration already existed. It basically says that you would look at your subordinates ONLY when you were totally busy each time you had allocation to do.

• So, if I am busy 50% of the time and have had to allocate 100 different SIs, I would have consulted a new subordinate 50 times at a cost of M each time.

• It appears to be looking only at management load, not the load in completing tasks. This is pretty sketchy, so it is hard to be sure.

Coupon collectors problem

• They don’t want the agents to always have to check all others to see if they should change links.

• They don’t want to evaluate which ones are better to evaluate.

• They use the coupon collectors idea to say that if they pick enough agents randomly they don’t have to be more sophisticated in making decisions.

Dealing with dynamic organization• When new agents join the system, it isn’t clear how

to include them.• They put them in a special list that is only looked at if

agents need more help. Then they select from this list based on what skills they are short (in waiting list). This is a good idea, but makes me wonder if there are other clever things they haven’t told us about. It wasn’t clear from earlier sections that they were being this sophisticated. Also, if there is no waiting list, this isn’t going to work well. Wouldn’t it be better if it knew what it had been short in the past?

hint

• When I can’t see the graphs clearly (from the printed page), they are always better in the pdf.