Support Center for Child Advocates
MEMORANDUM
Date: January 12, 2011
From: Frank P. Cervone and M. Christine Kenty
To: QIC-ChildRep Team
Subject: RFP Questions
Colleagues -- We have been studying the recently-issued QIC-ChildRep RFP with our local court administration. We commend the effort and recognize the challenges implicit in designing an empirical model in our complex field. Respectfully, the following questions related to the RFP and the R&D sites have emerged in our consideration:
Multidisciplinary service: The December 2010 notice of “RFP Posting Delayed” stated that “we will not include a multidisciplinary model of child representation at this time”. However the RFP makes no mention of multidisciplinary models, nor provides any indication about how to account for tasks or functions performed on behalf of the child’s lawyer by in-house colleagues such as a paralegal or “child advocate social worker”. Will these arrangements be permitted in the demonstration project? For example, consider that in some offices, the child advocate social worker (or perhaps “investigator”) performs the home visit on behalf of the lawyer, as part of relationship-building, “meeting the child” and fact-gathering. Of course, the ABA Standards encourage this approach. Most of the big-city children’s law offices (e.g., New York, Chicago, Los Angeles and Philadelphia) use social workers extensively in their programs. Please provide details on the validity of as well as how to approach these activities, in both service delivery and data collection.
Even though we do not require multidisciplinary service as a discrete treatment group as we once anticipated we would, we do not mean to exclude multidisciplinary representation of children from the treatment group. We would permit the arrangements you describe in the demonstration group. An application using what we called T2 in our earlier renditions of the R&D Criteria, where children would be represented by a multidisciplinary team would be acceptable. We have determined, however, that doing both T1 and T2 in the same jurisdiction would be problematic methodologically.
Numbers and Impact: Our big-city court has between 100 and 110 “private” lawyers on its “wheel”, available for court appointment in representation of children; virtually all are solo practitioners, and many also represent parents by court appointment. However 2/3 of the child representation is provided by two large children’s law offices, which function under the ABA Standards and thus seem ineligible for the control group (that is, how would a project show difference between control and treatment, if control is already using ABA model?). Assuming a minimum of 100 of the “private” lawyers were to participate, split randomly into control and treatment groups, and receive the requisite 25 cases minimum, can we expect that the reviewers will find this number sufficient “to detect a hypothesized program impact” as described in Section 3.3? Is there any circumstance in which you would accept 50 lawyers total, each with 50 cases?
No. Since the evaluation is analyzing lawyer behavior (which is expected to vary) in addition to the impact of the lawyer’s behavior on child outcomes, 25 treatment and 25 comparison lawyers is most likely not a sufficient number of lawyers to detect impacts, regardless of caseload.
Case = Child OR Family: Our jurisdiction counts a child as a case, but compensates children’s lawyers per sibling group. The RFP states that cases will be counted according to “whatever is the case definition in the jurisdiction, although we do not anticipate separating sibling groups in the random assignment.” Will the 4 siblings who are represented by the same lawyer, be counted as 1 case or 4 cases in the power analysis?
The 4 sibs would be counted as 1 case.
Size of Jurisdiction: What types of sites do you envision being able to qualify? What types of jurisdictions have as many as 2500 new cases in a two-year period? Or 100+ lawyers (more, considering attrition)? It seems the RFP is tilted toward statewide and big-city projects. If statewide, it seems unlikely that more than a precious few states, if any, will be able to standardize practice sufficiently to meet the test. In the several big-city programs, what will be effect of large offices counting as “one practice” in the power analysis (that is, these would appear to need thousands of cases)?
We realize that aspiring to a gold standard of empiricism may limit the number of jurisdictions that can meet our research requirements. We think there are jurisdictions that could participate, however. You are right that large offices practicing together would count as one practice and would need a very large number of cases of meet our research needs.
Blindness #1 – Impact of Exposure: Is it anticipated that the control and treatment groups have no communication with each other? If not, isn’t cross-pollution likely? For example, inside a given jurisdiction, members of the control group may change/upgrade their standard practice, or else risk practicing, during the term of the study, in the ‘second class representation’ club? Kind of like knowing you are taking a placebo!
We anticipate some spill-over effect and would seek to minimize it. For example, this is why groups of attorneys in a single practice will be consider together in the randomization process that deterimines who is in the intervention or control group. Treating all attorneys in a practice the same will help reduce spill-over effect.
Blindness # 2 – Impact of Data Collection: Is the data collection function expected to have no impact on the behavior of the control group members? For example, asking about meetings and phone contacts may prompt these activities; indeed reporting is often viewed as a key motivator of performance. Also, is the R&D site expected to perform quality control on the data collection of the control and treatment groups? While “supervision in the QIC model” seems appropriate to the research design, “supervision” for the control group will likely be a new, influential component of control group practice.
The use of the web-based case management system may very well change the practice of the control group of attorneys. However, we expect that the intervention will generate a large enough impact to allow us to detect differences between the control and treatment groups. In addition, we weighed this choice against the risk of poor quality data and low response rates we would expect with other data collection options.
We expect the site to see a benefit to using the web-based case management across both groups and to perform quality control on its use across both groups.
Consistency of Data Collection: We have concerns about the R&D site’s dependence on attorneys continuously and consistently collecting data, particularly for the control group. We have found it is very difficult to get even motivated people to collect data consistently, and this on fields that are relevant to their practice. What will be acceptable levels of oversight, subjectivity and self-adjustment by the participant lawyers?
We share your concern on this point which is why we intend to require control and treatment lawyers to use a web-based case management system (which we think they will actually find helpful to them). CT, for example, ties payment to use of the system. A jurisdiction able to do that would be a very attractive partner to us.