Guidelines for Area TPC Chairs (TPC Group Leaders)

IEEE INFOCOM 2011

April 10 - 15, 2011, Shanghai, China

Questions? Please email TPC Chairs at info11.chairs@gmail.com

Disclaimer: This document and the process described are subject to change without notice!
Please read the following if you are an Area TPC Chair (TPC Group Leader) for INFOCOM 2011.

The regular TPC member guidelines are at: Guidelines for TPC Members.

Non-TPC paper reviewers guidelines are at: Guidelines for Reviewers .

Meetings pertinent to Area TPC Chairs:

  1. Plan to attend INFOCOM 2011 TPC Informational/Planning Meeting on March 17, 2010 during INFOCOM 2010 (Highly Recommended).

  2. INFOCOM 2011 TPC Meeting: November 13, 2010. Location: Los Angeles, CA. Please make your utmost attempt to attend this meeting.

Summary of Action Items for Area TPC Chairs

Here are some guidelines to help clarify the role of ATPC Chair (or TPC Group Leader in EDAS parlance) for INFOCOM 2011.
  1. Every submitted paper is assigned an Area TPC-chair, a TPC-lead, and a total of three TPC members (including the TPC-lead). The TPC-chairs will load-balance assignments among the Area TPC Chairs to ensure approx. uniform load.

  2. ATPC Chairs should decide on "Early reject" in only two cases: (1) Paper is not ready for review (missing figures, clearly out of scope, etc.) and (2) Inconsistency between author names/affiliations on the paper and in EDAS. Early reject information should be sent to the TPC Chairs by the deadline (August 14, 2010).
  3. The paper review process starts from Aug. 22 and ends by Oct. 3. During this time period, Area TPC Chairs are supposed to verify that each uploaded review is of a good quality. If you see a superficial review uploaded, please contact the reviewer and ask him/her to improve the review (you can point towards specific issues). Three days before the review deadline (i.e., Sept. 30), Area TPC Chairs should contact the reviewers who have not finished their reviews and make sure they intend to do so on time. This is also a good time to nominate another (fourth) reviewer if one or more of the following holds (a) for some reason, one of the reviewers is not responding; (b) you see that 1-2 reviews for a certain paper are superficial or the reviewers marked a low familiarity score for the paper; (c) you already identify controversy among the reviewers. The best would be that you find an expert reviewer (e.g., using the reference list in the paper). We also have a group of backup TPC members (who have not been given assignments in the beginning) to whom you can assign a paper if you don't find an expert. Please ask the new reviewer to finish the review by Oct. 17, because you need the review when you make your recommendation.
  4. After the initial round of TPC member provided reviews are done, online discussion phase begins (as in previous years). During the discussion process, the TPC-leads are responsible for ensuring that the review scores are compatible with the supporting arguments in the written reviews, and followup on short reviews that provide little feedback. After online discussion, the TPC-leads will place each paper into one of 4 categories: Accept, Reject, Discuss, Needs Extra Review. The last category could apply due to non-authoritativeness of reviews (expertise) or genuine controversy. Area TPC Chairs are supposed to oversee this online discussion process. If you don't see a discussion, please start it and ask the reviewers to participate.

  5. The papers in each Area (TPC group) need the attention of the Area TPC chairs in a fairly short period (2-3 weeks). Your availability during this period is critical for timely and successful completion of the process. Assuming that there are 2000 submissions and 40 ATPC Chairs, there will be 50 papers to handle per ATPC member. Of this, the TPC Chairs will review the bottom 50% of papers in each group (i.e. approx 25 papers) and reject them (for most of them). Thus, we anticipate that each Area TPC chair will have to actively deal with 25-30 papers. Some will be clear accepts with the area chairs will quickly recommend/confirm, and most papers will be in the Discuss/Needs Extra Review categories (but could also include some "weaker" accept/rejects with conflicting scores as well). Based upon their expertise, the Area TPC Chairs could choose to reject some more papers without an extra review. About 15-20 papers may require an extra review to help the decision making.

  6. The Area TPC chairs can use the Backup TPC members (who have not been given assignments in the beginning) to get one extra review per paper. The list of Backup TPC members is sent to the Area TPC Chairs by email (please contact the TPC Chairs if you do not receive this information by 10/11/10). Area TPC Chairs might also find it very useful to have a list of pre-identified RELIABLE REVIEWERS (who can handle 1-2 papers at short notice) and notify them in Aug/Sept. 2010 that their services might be needed in October.

  7. Based upon the extra review, the Area TPC chair can recommend reject (if the review confirms/clarifies other reject recommendations), recommend accept (if the consensus is very strong) or recommend discussion at the TPC meeting. We encourage you to recommend reject decisively based upon the extra reviews to reduce the decision load for the TPC meeting. While you have authority to recommend ACCEPTs, we urge some caution because of the limited number of total accepts and the need for the TPC meeting to weigh in if papers do not have clear consensus for accept.

  8. We expect the Area TPC chairs to attend the TPC meeting and lead/co-lead the groups having papers for which they were the Area TPC chair. We will also attempt to match TPC members to the groups so that they can participate in at least a few of the papers they reviewed. This way, the few minutes spent upon the paper at the TPC meeting will be more productive and the decisions will be better informed. If the Area TPC chair cannot attend the meeting, we request comments for each paper in the final discuss category to guide the TPC meeting groups.

  9. TIMELINE

    Please see http://cse.unl.edu/~byrav/INFOCOM2011/

Frequently Asked Questions

#1 Who are the ATPC Chairs for INFOCOM 2011?

The complete list of ATPC Chairs is available online on the conference website (here)

#2 What are the different TPC Groups in EDAS?

Area TPC Chairs are also referred to as TPC Group Leaders in the EDAS system. We will have 40 TPC Groups, while each TPC Group will be associated with an Area TPC Chair after all the papers are submitted. The TPC Chairs will take care of this assignment to ensure load balance. Authors will be allowed to select an AREA to associate their paper with at the time of paper submission. Once the papers have been submitted, some papers with different AREAs may be combined and papers moved across groups to balance the load across TPC groups.

During the INFOCOM TPC Meeting in November, these TPC groups will also be the basis for the meeting's discussion groups. More details on the TPC meeting logistics will be provided later.

The different AREAs are:

On EDAS, the TPC groups are numbered as TPC Group 1, TPC Group 2 ... etc. Each ATPC Chair is a TPC Group leader for one TPC Group in EDAS. As mentioned earlier, the TPC groups may not strictly correspond to the areas listed above in order to balance load across groups. Papers from a single AREA may be split across multiple TPC groups or papers from multiple AREAs may be merged into a single TPC group. Papers may also be moved across groups to handle Conflict-of-Interest issues with the ATPC chair handling the group.

#3 What are the deliverables for an ATPC chair and when are they due?

The deliverables from the ATPC chairs are two-fold and due August 14, 2010 and October 31, 2010.

  1. Early Reject Phase ATPC Chairs should decide on "Early reject" in only two cases: (1) Paper is not ready for review (missing figures, clearly out of scope, etc.) and (2) Inconsistency between author names/affiliations on the paper and in EDAS. Such inconsistency may result in assigning papers to reviewers with conflict of interest.
  2. Review phase
    1. A summary spreadsheet with your decisions on each paper in your group. Please include other fields in your spreadsheet such as EDAS ID, Title, review average, span along with the your decision, as necessary and
    2. Your comments which need to be entered on EDAS Notes section for each paper

#4 How to access my INFOCOM 2011 Groupleader (ATPC Chair) page on EDAS?

Please note that when you log in to EDAS, you will need to select the TPC Groupleader role for INFOCOM 2011 apart from any other roles (author etc.) you may have. To do this, please access http://edas.info and click on the "EDAS login" button. and enter your email address and password. If you have forgotten your password, click on the link to email yourself your password. The link is
http://edas.info/emailPassword.php Then click on "My TPCs" tab. You will see a list of conferences in which you serve as a TPC member. Please click on "INFOCOM 2011", in the row where your Role is listed as "groupleader." You should see the tabs marked "Home", "Papers", "People", "Reviews" etc. at the top. This is your INFOCOM 2011 Groupleader page for INFOCOM 2011.

#5 Actions using the Groupleader (ATPC Chair) page on EDAS

All of these actions will need to be carried out from the Groupleader page. To access your Groupleader page, please see instructions above.
  1. Listing all the papers in my group

    Please click on the "Papers" and then click on "List". By default, all papers with "active" status and the following fields: Paper number, Paper title, Conference name are selected. Please add other useful fields to include such as "TPC group", "Authors with affiliation and country" and "Number of reviews of all types completed." Then click on the List papers button. You will be presented a list of all the papers in your group with the appropriate fields.

    Please note that a paper on EDAS can only be in one of the following different (mutually exclusive) status categories.

    • Pending (registered, but no manuscript)
    • Active (registered, with manuscript)
    • Withdrawn by author or chair
    • Need reviews
    • Being discussed
    • Accepted for publication
    • Published
    • Rejected
    If you need to list all papers, use Papers->List->Check all boxes

  2. Checking for conflicts of interest for papers in my group

    Please list all the papers as described above and verify that you do not have a conflict of interest (see description below) with any of the authors of each paper. If you do have a Conflict of Interest, please inform the Program chairs by email mentioning the paper title and paper number. Please review the INFOCOM 2011 Definition of Conflict of Interest for determining the set of individuals you need to list on EDAS. People/institutions that you are in "conflict of interest" consist of the following:

    • Same institution within 4 years (anywhere in your organization or parent company counts); note that this includes your previous or future institution if you have recently changed job, or are in the process of changing job.
    • Co-author (or pending) a publication within 4 years, in any technical area, including patents, tech. reports, etc.
    • Active or pending grant, proposal, or application of any sort together
    • Ph.D, post-doctoral advisor/advisee at any time, current or past
    • Close personal friend (or enemy!)
    • Any other non-technical bias

  3. Reading the reviews and discussion messages for a paper in my group

    From your Groupleader (Area TPC Chair) page, please click on the Papers->List tab. Then please click on "List papers" button after checking the default boxes - Status to include "Active", Fields to include "Paper number", "Paper title", "Conference name" You may also check the following boxes "Number of TPC reviews completed" and "Number of regular reviews completed" You will then be presented a page with the EDAS ID and title of all the papers in your group. You may click on the EDAS number for a paper (in the first column) to see the paper reviews, TPC summary and rating as well as the discussion messages among the TPC lead reviewer and regular reviewers for that paper.

  4. Ranking the papers according to the TPC summary ratings in my group

    While listing the papers as discussed above, check also the box marked "Weighted average of TPC scores: Recommendation." Once you are presented with a list of papers in your group, click on the column heading "TPC Avg." Now, you will find the list of papers sorted by the TPC summary rating. (4.0=Accept; 3.0=Needs additional review; 2.0=Discuss at TPC meeting; 1.0=Reject; 0.0=TPC review/summary missing).

  5. Dealing with papers with missing regular reviews and missing TPC summaries.

    Most papers in your group should have 3 Regular reviews (Review 1, 2, and 3; from Reviewers A, B, and C). Most papers should also have 1 TPC Review/summary (Review 1) i.e. Lead Reviewer (TPC) summary of discussion phase with a Recommendation. We are reminding reviewers and TPC members to submit the reviews soon. In addition, we are assigning backup TPC members and designated non-TPC member reviewers to these papers for urgent reviews. However, if there are less than 3 regular reviews and/or no TPC summary review for a paper, and/or a paper is marked "Need additional review" after the discussion phase, you may choose to assign an additional review to the paper. Please see below on how to assign a regular reviewer or TPC reviewer.

  6. Assigning an additional review for a paper

    To assign a regular review for a paper, please click on the link marked "+ Assign another review" You may assign the paper to any qualified reviewer on the EDAS system by typing in a last name and clicking on the "Find" button. Please avoid regular "TPC members" for the conference and any others with conflict of interest with the authors (you may need to check with the potential reviewer first) and any others who are listed as Non-preferred reviewers by the authors. Please do not assign any review to a Backup TPC member if you see a number greater than 6 in parantheses next to their name when you use the pull-down menu to add another reviewer using "+ Assign another review." You may add other additional reviewers (not on the Backup TPC list) at any time.

    To assign a TPC review for a paper, please click on the link marked "+ Assign another TPC" at the bottom of the regular reviews for a paper. You may assign a TPC review to yourself and summarize the existing reviews and online discussions and come up with a TPC Recommendation.

  7. Providing ATPC Recommendations (to TPC Co-chairs) for papers in my group

    Your comments need to be entered on EDAS Notes section for each paper (see detailed instructions below). After you have entered your comments, found extra reviews etc, please prepare a SUMMARY SPREADSHEET of paper #s with your proposed decision for your papers.

  8. Preparing for the TPC meeting

    Following the ATPC Chair guidelines, please suggest 10-15 papers (exact number to be provided later) in your group for discussion at the TPC meeting. If you are planning to attend the meeting, you will be asked to lead the discussions of these papers to an audience which will include the TPC/regular reviewers (TPC members) for the paper. If you are not planning to attend the meeting, please have information available in a Powerpoint presentation (and/or spreadsheet) which summarizes the discussion points for each of the paper in the Discuss category including the average review score, strengths/weaknesses etc.

  9. Participating in online discussion with the TPC lead reviewer and regular reviewers for a paper

    There is two options in EDAS for this.

    • METHOD 1 : Simply click on the bubble icon for Discussion [Add discussion item] at the bottom of the page. Any comments you type in will be sent anonymously to all TPC lead member and TPC/other reviewers of the paper by email as well as show up on EDAS. You will be able to see the online discussion messages on the EDAS site for each paper. However, you will not receive the online discussion messages by email (as the reviewers would).
    • METHOD 2: Please pull up the paper details by clicking on the paper number. Then add yourself as a regular reviewer by clicking on "Assign another review" Click on the "Notify" button at the bottom of the page. You should see your name in the list of regular reviewers for this paper. Then click on the bubble icon for Discussion [Add discussion item] at the bottom of the page. Any comments you type in will be sent anonymously to all TPC lead member and TPC/other reviewers of the paper by email as well as show up on EDAS. Please make sure you don't reveal reviewer identities in your message since the reviewers don't know each other's identities.

#6 Where can I find more detailed step-by-step instructions on the ATPC Chair EDAS interface?

Please see below.

ATPC Web page Steps:
===================
1. Login on EDAS
2. Click on "My TPCs" (yellow colored Tab on the top of the page)
3. Click on "INFOCOM 2011" link.
        {NOTE: You may see a line like this: ignore it & follow
                instructions below! "Currently not reviewing any papers
for this conference."}
4. Click on the "Papers" Tab
        {NOTE: not to be confused with "My papers"}
5. Click on the "List" Tab (under "Papers")

--------------------------------------------------
Useful things to tick in EDAS paper "list" page:
        {NOTE: there are several similar looking lines: make sure you
                check the correct ones}

[] Active (registered, with manuscript)

[] Sequence number
[] Paper number
[] Paper title
[] Authors with affiliation and country
[] TPC group

[] Number of regular reviews completed
        {NOTE: if only 2 reviews have been completed, we have usually
          assigned a backup TPC member for extra review already}

[] Span (max-min) of weighted review scores
        {NOTE: large span => higher variance in reviews}
[] Weighted average of TPC scores: Recommendation
        {NOTE: 4 = Accept, 3=Needs Rev, 2=Discuss, 1 = Reject; 0.0 => NOT
         COMPLETED by TPC Lead}
[] Weighted average of non-TPC review scores: Recommendation
        {NOTE: IMPORTANT: Sort by this field once you see the list
                papers. You do not need to look at papers below the reject
                threshold,  unless in rare cases, the tpc discussion ("TPC
                Avg" column entry) recommends discuss or needs extra
                review (2.0 or 3.0). }
--------------------------------------------------

6. Click on "List Papers" Button to see the papers list. You should
  have around 50 papers in your list.

7. Sort by the column "Non-TPC Avg.".
        - For papers with avg rating >= ACCEPT-THRESHOLD (straight accept), we want
          you to affirm the decision on your spreadsheet.
        - For papers with avg rating <= REJECT-THRESHOLD (straight reject), you do
          not have to process them, except  in rare cases,
          when the tpc discussion ("TPC Avg" column entry) recommends
          discuss or needs extra review (2.0 or 3.0). }
        - the remaining papers ALL require your input, and possibly
          (if you judge so), an extra review.


8. Writing your input/Assigning a reviewer on EDAS:
        - Click on the paper number (or open it in a new window/tab)
        - You can see the reviews for the papers, TPC discussions and summary
        - To fill out the review form for your input, click on Edit review 
           <pencil icon>. This Icon is near the "Notes" row (above the   
           "Printing problems" row)
        - To assign a reviewer, click on " (+)  Assign another review"
Thanks again for your valuable contribution to INFOCOM 2011. We look forward to working with you to make the program a success.
Byrav Ramamurthy, Jie Wu and Qian Zhang
INFOCOM 2011 TPC Chairs