As program chair for the 1991 OOPSLA conference, I decided to address this situation proactively by writing this article. The purpose of this article is to provide advice to prospective OOPSLA paper authors, based on my past experience. If you are a prospective author of an OOPSLA paper, I hope you will be able to use this advice to make improvements to your paper before submitting it to the program committee, thus improving its chances for acceptance.
This article will also offer some suggestions to help you decide whether to submit a paper to OOPSLA at all. Members of the program committee spend a considerable amount of time reviewing a large number of papers. You should avoid unnecessarily adding to this workload, by not submitting a paper that is clearly inappropriate.
The primary requirement is that your paper must contribute to our understanding of object technology. It must have something new to say, and its message must be of sufficient importance and interest to warrant the attention of the OOPSLA community.
If your paper is a research paper, it should describe a new idea or a new technique. It must describe original work. Your paper should present supporting evidence, not just conjecture. Idea papers should be backed up by a convincing analysis.
If your paper is an experience paper, it should present new data based on actual experience that demonstrates the effectiveness (or ineffectiveness) of object technology, describes problems encountered, and makes suggestions for improvements. It should provide new evidence, either positive or negative, to evaluate existing ideas or techniques. It should help provide direction for future research, as well as provide new insights of value to other practitioners using object technology.
The distinction between research and experience papers is not absolute. It is possible to gain experience in a research setting, and it is possible to develop new ideas or techniques while applying a technology. However, if you focus on one of these two aspects, you will write a better paper.
Your paper should make its contribution abundantly clear. It is not the job of the program committee to ferret out this information. One aspect of identifying the contribution is to cite and make appropriate comparisons with previous work. A research paper should compare and contrast the work with prior work, demonstrating novelty. An experience paper should compare its results with other papers that present similar or opposing data. An experience paper that merely confirms that which is well known has opposed to that which is widely believed) is of little value.
Obviously, your paper must not have been published elsewhere in the same or similar form. Less obviously, your paper must not be under consideration elsewhere in the same or similar form. While the desire for authors to "hedge their bets" by submitting the same or similar paper to multiple conferences and/or journals is understandable, this practice places an unnecessary burden on the peer review process. It can also cause embarrassment or confusion if the same paper is accepted at more than one conference. If you believe that unusual circumstances warrant simultaneous submission to more than one forum, you MUST notify ALL the program chairs and editors involved; simultaneous submission without notice is considered highly unethical. (Note that archival journals frequently accept versions of papers that have already appeared in conferences, but usually it is in your best interest to get feedback from the conference presentation before submitting a polished and revised version to a journal.) It is also considered inappropriate to submit multiple papers to the same conference that cover substantially similar or overlapping material. (It may be appropriate, however, for multiple papers to be submitted concerning distinct aspects of a single project, particularly if the various papers have different authors.)
If you have published previous papers on the same subject, your paper should clearly indicate the relationship between your new paper and the previous papers.
Finally, your paper must be well written. It must clearly communicate its message to the intended audience.
The first question you should answer is "Why am I writing this paper?" If the answer is of the form "to document what I have been doing for the past two years", then you are in danger of writing a bad paper. It may be a shock to your pride, but most people are not interested in what you have done for the past two years. If you need to document your efforts, write a memo or a tech report. Another poor answer is "to help build my case for tenure". Tenure may be your initial motivation for writing a paper, but it should not be the only motivation.
The purpose of your paper should be to communicate something to someone. So, the next questions are "What is my paper trying to say?" and '"Who is the audience for my paper?" If you cannot clearly answer these questions, then the paper is likely to be poor.
A focused paper is better than a scattered paper. Resist the temptation to describe every great idea you had while working on your project. Pick a primary message and communicate it well.
After deciding what the paper is trying to say, the next question to answer is "Is it worth saying" Is it a new message, or just a rehash of an old message? Is the message of value, or potential value, or is it trivial? Is it conjecture, or have you demonstrated the soundness of your conclusions?
After deciding who the audience is, the next question to answer is "'Is OOPSLA a good way to reach my audience?" Does my message have value to researchers and developers of object technology? Does my message have value to practitioners using object technology?
An experience paper may be called premature if it offers conjectures about expected results rather than reporting observed results. For example, if you are just about to release a product, it is premature to claim improvements in software maintenance until the product undergoes a maintenance cycle. If you have just created a "reusable" library, it is premature to declare it reusable until actual reuse occurs.
One area of difficulty is portability. Experience has shown that designing a portable system of any kind can be surprisingly difficult. One cannot convincingly claim to have designed a portable system unless the portability of that system has been tested in at least two environments.
The decision to accept or reject a paper that is premature is a judgment call by the program committee. A committee may choose in some cases to accept a paper that presents early work of a profound or provocative nature. However, you should honestly evaluate the maturity of your work before deciding to submit a paper on it. You may be able to write a much stronger paper for next year's conference!
In addition to being convincing, a proof must prove something worth proving. It is not worth anyone's time to read a paper that proves an irrelevant result. Be careful about including a proof in an effort to make your paper more "prestigious". This approach may backfire, as a sloppy or unmotivated proof can easily cause a paper to be rejected that otherwise might have been accepted.
I will explore this question in the context of object-oriented programming languages, although it applies to other areas as well. Many papers submitted to OOPSLA are written in the context of a specific object-oriented programming language. In many cases, the papers are actually addressing more general issues.
As a hypothetical example, suppose you want to write a paper on "extending the Foo programming language to support distributed computing". A paper on this subject could easily address many issues that are not specific to the Foo programming language, although some might be. Such general issues might include naming, storage management, concurrency, and distributed schema (type) management. Your paper would be much stronger, and much more likely to be accepted, if it addresses these issues in general terms wherever possible, using the Foo-specific work as examples. If your paper is inherently Foo-specific, then you should consider submitting it to a more appropriate forum, such as the Foo Conference, the Foo Journal, or a meeting of the Foo Users Group.
A gray area arises if your paper deals with a distinctive aspect of the Foo language. A paper that can demonstrate the value (or disadvantage) of a language-specific feature could be of great interest to all language researchers.
Most authors will benefit from having their paper reviewed by a skilled writer. If your native language is not English, you have an extra burden. If at all possible, try to have your paper reviewed by a native or fluent speaker of English.
Do not hesitate to attach a cover letter to your paper if there is additional information that would be useful to the program committee. For example, if you have published previous papers on the same subject, you might attach a cover letter to explain in more detail the novel contributions of the paper you are submitting.
Finally, as has been noted elsewhere [Wegman], the conference review process is necessarily imperfect. The reviewers operate under strict time constraints, and the committee must make quick decisions. A paper will not receive the careful attention that it would from a journal. Furthermore, the committee may need to satisfy other constraints in putting together a successful program. As a result, some good papers will be rejected. Authors should carefully consider any reviewer comments and get opinions from experienced colleagues before deciding whether to abandon the effort or to revise the paper and submit it elsewhere.
Additional insight can be obtained from the excellent paper by Smith [Smith] that describes the review process from the point of view of the referee.
[Wegman] Mark N. Wegman. "What it's like to be a POPL Referee, or How to write an extended abstract so that it is more likely to be accepted". Sigplan Notices 21:5 (May 1986),91-95.
Alan Snyder Hewlett-Packard Laboratories P.O. Box 10490 Palo Alto CA 94303-0969This information last updated by ayers@zti.com Mon Jan 1 12:05PM 1995