Hello, I am Maggie Miller, the principal of Maggie Miller Consulting.  I conduct program evaluation for small to mid-size nonprofits in the Denver/Boulder area.  Sometimes I think I should change that tagline because I’ve gotten referrals to larger nonprofits (whom I adore), but I am still very attached to the small identity.

Hot Tips:

• Trust that being small can be an advantage: when people hire you, they do so because you’re you; you’re not going to be delegating to staff.
• If you’re really small, you won’t have employees, so…
• Have subcontractors instead.  I work with some wonderful people who do data entry, multicultural interviews, and web development.
• Try to get some work as part of a team.  This will give you the camaraderie and learning opportunities that you would have if you worked for a larger company.
  • Know what you’re good at; know what has your name on it.
  • Know your limits:
  • When asked, “Oh, you consult with nonprofits…do you do strategic planning?” the answer is NO.  (Unless you do that too.) Get to know consultants who do related work, and make referrals.
  • Don’t take jobs that are too big.  Be happy to refer people to the larger shops in town.
    • Try to get one gig that brings in 40%-60% of your income, then you can be free to help smaller clients with the rest of your time.

Lesson Learned:

• The better you know what you can and cannot do – and what you like to do and don’t like to do – the better able you will be to attract your perfect clients.
• It’s all about relationships.  Enjoy them and keep them strong.

Rad Resources:

• This book: Attracting Perfect Customers: The Power of Strategic Synchronicity by Stacey Hall and Jan Brogniez.  This is somewhat woo-woo (i.e., law of attraction) but has guided my business philosophy and is full of useful exercises that brought me wonderful clients!
• All of the American Evaluation Association’s (AEA’s) amazing and affordable educational offerings, and fabulous resources like Stephanie Evergreen’s blog and Ann Emery’s Excel videos.
• See other AEA365 posts on Small is Beautiful

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. 

data visualization · Independent Consulting

I’m Gail Barrington, an independent consultant with more than 25 years of experience in program evaluation and applied research, and a published author. I’m a senior evaluation adviser to a number of organizations, I teach on-line, present workshops and webinars, and write as much as I can. Our clients hire us for our expertise and specialized skills but they keep us for the relationships we build and the knowledge we acquire about their organizations. This provides a golden opportunity to think more deeply about translating evaluation findings into action. Don’t submit your report and walk away. Our impact begins when we plan for KT. My freely re-interpreted version of the Canadian Institutes of Health Research’s definition of KT is the exchange, synthesis and application of research findings by evaluators and their clients to accelerate the effectiveness of evaluated services, products and systems.

Lessons Learned:

• Plan for KT activities at the beginning of your project and make them a line item in your budget.
• Report early findings and planned KT processes in your interim reports and obtain stakeholder feedback.
• Reflect on your final report with your client and other stakeholders and determine what information should be disseminated, what audiences reached, and what strategies and mechanisms used.
• Once the KT phase is complete, evaluate its effectiveness too.

Rad Resources:

• Dr. Melanie Barwick, a presenter AEA 2011, developed a Knowledge Translation Research Plan Template and a Training Manual to help you plan for potential KT strategies when writing your proposal or evaluation plan.
• Barwick also reviewed KT practices in an extensive number of medical research articles and drew some interesting conclusions. While we tend to favor conference presentations and papers, Barwick found their effect to be mixed at best. The most effective KT interventions include:
  • Interactive small group learning
  • Educational outreach
  • Electronic and poster reminders
  • Computer prompting systems and decision support
  • Multi-professional collaboration and teamwork.

With this kind of information, we can strategize with our clients about effective next steps. Because we work closely with them and because they trust us, we can encourage them to develop KT plans and strategies. Working together we can grow the impact of our evaluation results exponentially.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

dissemination

I’m Jean Eells and I’ve operated E Resources Group, my evaluation and consulting business, for 14 years.  My clients are small non-profits and small local or state governmental departments and I run from 8 – 12 contracts at a time and up to 15 in a year mostly by myself, or with subcontractors.  My tip is about getting paid and keeping cash flow timely which is essential to be successful as a small independent evaluation firm.  As a small shop, I can’t wait months for a paycheck and I’ve learned a way to avoid most of the stomach-churning delays I hadn’t anticipated.

Lesson Learned:  Whenever I enter into a contract with a new client, or one who has undergone significant restructuring since I last worked with them, I always write the payment schedule to include a nominal payment right at the beginning before I’ve produced any real products for them.  Sometimes the contract language I use is “due payable upon contract signing” and the amount may be $500 or $1,000.  Most of the rest of the payment schedule is tied to benchmarks common to most consulting contracts.  For state contracts which will only pay in reimbursement, I negotiate for a small easily achievable benchmark for the same purpose.

Hot Tips: Getting the client’s organization to enter me into their accounting system often takes longer than they assure me it will take! This also gives me a heads up to anticipate the length of time it may take from when I send an invoice to when I have the check in hand.  It allows my client – who is not the same as the accounting department – to save face when the check gets delayed for weeks and they don’t have to be embarrassed when their organization doesn’t pay as quickly as they said. I can’t say the technique makes slow-paying clients hurry any faster but it helps me manage my expectations and thus my cash flow more effectively.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

cash flow

My name is Susan Wolfe, the owner of Susan Wolfe and Associates, LLC, a consulting firm that applies Community Psychology principles and evaluation skills to strengthening organizations and communities.  Being a “Jack of All Trades” is one way to combine the skills and core competencies of evaluation and community psychology to be successful as a small independent evaluation firm.

Lessons Learned: The core competencies for Community Psychology include program development, implementation, and management; organizational, collaboration, and coalition development; community organizing and advocacy; policy analysis; and information dissemination.

By combining Community Psychology and evaluation competencies I can offer a greater range of services, including strategic planning, grant writing, facilitation, community advocacy, and organizational capacity building.  There are pros and cons to this.

• Pros: Diversifying my business means I am not reliant upon a single contract or line of services. Having an understanding of program development, implementation, and management helps me design evaluations that are more realistic based on a deep understanding of programmatic and organizational constraints.
• Cons:  Maintaining boundaries across activities is difficult.  It means being careful not to influence what you are evaluating and maintaining the distance necessary to remain objective when you have been involved in project development.

Hot Tips: These tips may help you expand your current set of evaluation skills so you can successfully offer a broader scope of services to existing and potential and strengthen your evaluation services.

• Look outside the evaluation field for workshops or certification programs to develop new skills.
• Identify the skills gaps in your area.  Is there a shortage of people who can facilitate development of a strategic plan or write grants that get funded?
• Use your other skills to enter new areas where evaluation work might be available. If you are writing a program grant, will it require an external evaluator?
• The logic model you developed for your evaluation may be a powerful tool for facilitating a staff retreat.  Consider using it to communicate with staff about the evaluation and increase their participation in evaluation activities.

Rad Resource:  For more information about Community Psychology, click here.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

psychology

Hello, I am Matt Feldmann, the principal researcher and owner of Goshen Education Consulting, Inc. We focus on educational evaluation for clients in Southern Illinois. Establishing a service or content niche is one way to be successful as a small independent evaluation firm. I define a niche as a clearly delineated specific opportunity for your evaluation talents that is currently underserved.

Hot Tips:

• Clearly identify yourself. Most organizations do this by defining a mission statement. This is important to provide a static understanding of: who you are, who you want to serve, and what you are good at (or want to be good at).

• Clearly identify your potential clients and their needs. This is done by developing a thorough understanding for the services they provide and operating within the framework of what drives your clients to succeed.

• Target your specific niche market. A specific niche market is that well researched opportunity that is not properly served by you or others and provides you the opportunity to expand your services to further meet your mission statement. A specific niche evaluation service will provide an evaluation solution to your client that exceeds their expectations and creates a “halo effect”, i.e., opens the door to additional opportunities to serve your clients and others like them.

• Get the word out. You will need to identify the marketing approach that is best for you, but remember you need to be very public (yet not obnoxious) about your evaluation solution.

• Expand the niche and identify new niches. By expanding one niche evaluation opportunity, you are given an unbelievable opportunity to learn more about your clients, to develop supportive relationships, and to learn more about how your services and talents could further benefit your clients.

Lesson Learned: Stick to your guns. You should not waiver on your service niche if you have properly identified yourself; your potential client’s needs, and have a clear understanding for their needs.

Rad Resources: The following will not be found in most evaluators libraries, but are invaluable resources that can easily be applied for evaluation organizational development and niche identification.

• Crossing the Chasm by Geoffrey A. Moore is an invaluable resource that can be easily applied to evaluation organizations that want to clearly specify and develop their services.
• Good to Great by Jim Collins provides organizations with seven researched ways to develop a solid organizational approach.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

clients · organizational factors

Hello, I am Leah Goldstein Moses, founder and CEO of the Improve Group and 2012 President of the Minnesota Evaluation Association. When I founded the Improve Group in 2000, I was learning to be a consultant at the same time I was refining my evaluation skills. My practice has grown from myself and a loose network of other independent consultants to a consulting firm of 18 staff.  Running a company is different than independent consulting. It took nearly 4 years before I had my first employee. From that first day, I had new responsibilities – making payroll, setting HR policies, and developing a new network of advisers and resources. I’ve learned a lot of lessons along the way on how to be successful as a ‘not so small’ independent evaluation firm.

Lessons Learned:

• Take strategic risks – but prepare for the consequences. As our company grew, the risks also grew. I now take a strategic approach to big decisions, such as a new hire or pursuing a large project. I ask myself: what if this doesn’t work out? What are the potential risks? What choices will we need to make in reaction to those risks?
• Find a focus and identity. We love evaluation. We also do strategic planning and research, but always with an underlying evaluation focus. We are known as evaluators but we decided to work across sectors.  So we do evaluation in arts, human services, formal education and informal learning, health, transportation, development, and corrections. We are experts in evaluation and have found ways to supplement our expertise in these sectors. You might find a different identity and focus – either in one sector, a set of methods, or something else. If you can describe who you are to your clients, collaborators, and community, you will be fine.

Rad Resources:

• Interested in learning more about growing and sustaining an evaluation business? Attend Improving Evaluation Practice Management During Chaotic Economic Times: Three CEOs Reflect on Strategic and Innovative Diversification, Budgeting, and Employee Support and Development at the AEA conference on Thursday, Oct 25, 4:30-6:00 PM, where I’ll be presenting with Gary Ciurczak, Richard Hezel and Samantha Hagel.
• I use the AEA365 blog, the mande listserv and the evaltalk listserv to stay fresh on current evaluation topics.
• I enjoyed the Momentum Effect by J.C. Larreche; a book that examines the factors that helped companies grow year after year.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

strategic planning

I’m Norma Martinez-Rubin, an independent evaluation consultant with a practice focused on evaluating disease prevention and health promotion projects. Small-sized independent evaluation consultancies can offer clients the accessibility and nimbleness absent in larger organizations mired in their own bureaucracies. Our multiple roles as Chief Executive Officer, Chief Operations Officer, Chief Informational Officer, and Chief Creative Officer require becoming informed about managing ourselves, our content, our service, and our client relationships. I hope the following lessons and resources will help you to be successful as a small independent evaluation firm.

Lesson Learned: Shifting career gears to launch a solo practice entails exploration, soul searching, and rethinking how to best balance personal life and professional interests. Hence, the attraction to sole proprietorship and its rewards: autonomy, schedule flexibility, and choice over the selection of work projects that coincide with personal values. Did you notice my exclusion of endless financial reward? That was intentional.

Going solo is among the riskiest career endeavors. I have learned that I must be ready to ride the financial waves associated with economic shifts and their effects on prospective clients. Contract work may not be as abundant when prospective clients are facing economic woes. The financial safety net (steady income, pre-packaged insurance and retirement benefits, paid travel, and professional development) offered by employers is absent. So, what to do? Short of obtaining a Masters in Business Administration degree, identifying a business mentor is of utmost necessity. Passion alone will not yield income to cover the costs of doing business.

Rad Resources:

• Perhaps not so “rad” because of its governmental roots, yet a helpful orientation to the ins and outs of establishing and growing a business, the U.S. Small Business Administration is a resource for starting and managing a business, contracting, and obtaining loans and certifications as business progresses.

• SCORE–previously known as the Service Corps of Retired Executives and supported by the SBA– offers counseling and mentoring.  Through the SBA or SCORE websites, one may set up free email news particular to a location.

• Social marketing options can provide visibility and broad reach for small businesses. As a service organization, it is of utmost importance to identify one’s value to prepare a strong online presence. Susan Chritton’s book, Personal Branding for Dummies presents a fun, provocative approach to identifying personal value as a service provider. After all, along with your content expertise, the core of your business is you. To thrive during the ups and downs of the business of independent consulting, developing that core strength is essential.

The American Evaluation Association is celebrating the Independent Consulting TIG (IC) Week. The contributions all week come from IC members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

branding · resources

Hello, my name is Nicole MartinRogers. I am the Survey Research Manager at Wilder Research, in Saint Paul, Minnesota. Wilder Research manages projects that range from small evaluations for local nonprofits to multiyear statewide evaluations of complex interventions.

Hot tip: Project management (the discipline) offers great tools that will help keep your projects on track, for example:

• Work Breakdown Structures, used to plan a project by breaking it down into smaller, more manageable tasks.
• Three-Point Estimation, a strategy for creating realistic cost estimates by accounting for variability using experience, historical information, and expert judgment to define an optimistic estimate, a pessimistic estimate, and a most likely estimate.
• Network diagrams and Gantt charts, tools that help illustrate a project timeline and contingencies between tasks, so you can easily show how a delay in one aspect of the project will result in changes to the entire project timeline.
• Agile or Scrum methods for regular team check-ins to keep your team and project on track.

Lessons learned: Don’t feel you have to adhere strictly to the templates and documentation associated with project management (the discipline). Instead, try out a few tools, take what works for you, and modify it to address your specific needs.

Try working with your team to develop a Work Breakdown Structure or network diagram. It will help them understand how their part fits into the big picture. Also, work with your client to finalize these documents, and get client sign off as a formal step in the project.

Team members, clients, and other project stakeholders are easier to manage and more satisfied when your evaluation project is completed on time and within budget, and when it delivers all of the agreed upon products. This can be achieved by carefully planning and managing resources and expectations throughout the course of the evaluation.

Rad Resource: Google is a great source for more information about any of the specific tools described above. Two books that are also helpful: PM Crash Course, Premier Edition: A Crash Course in Real-World Project Management, by Rita Mulcahy; and Project Management for Dummies, by Nick Graham.

Twin Cities Hot Tip: When you’re in town for the AEA conference this fall, you should check out some of Minnesota’s beautiful state parks. William O’Brien is less than an hour drive from the Twin Cities and has great hiking trails and views of the St. Croix River.

The American Evaluation Association is celebrating Minnesota Evaluation Association (MN EA) Affiliate Week with our colleagues in the MNEA AEA Affiliate. The contributions all this week to aea365 come from our MNEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

project management

Greetings from beautiful Boise! We are Rakesh Mohan and Amy Lorenzo of Idaho’s legislative Office of Performance Evaluations. For the past six years, our office has been conducting surveys solely in an electronic environment. This approach has worked so well that we have all but abandoned the use of paper surveys. When we looked at costs, both in postage and manpower, paper surveys just didn’t make sense any more. That is, until this year.

As part of our current study of barriers to postsecondary education, we sought input from members of the Idaho School Counselor Association. Initially we thought of contacting them via e-mail and requesting them to complete a web-based survey. However, once we learned that they would be participating in a statewide conference in our town, we decided to brief them about our study at the conference and asked them to complete a paper survey and turn in their completed surveys at the conference registration table. The response was phenomenal!

Over the next 30 minutes, attendees answered our survey questions, chatted with us about our study, and offered to meet with us again in the future. By the end of the conference, we had received over 70 surveys, a response rate of nearly 50 percent. In a couple of hours, we had accomplished what normally takes us several weeks to achieve. We were able to close out the survey and begin our analysis the same day we distributed it.

Hot Tips: Why did this approach work so well? In the spirit of full disclosure, we should note that attendees were offered an extra raffle ticket for turning in a survey. However, that was really just one small piece of the equation. What really made this process a success was twofold.

1. Conference attendees were a captive audience. By asking them to complete the survey while at the conference, we were no longer competing with their other professional and personal demands. Our e-mail wasn’t lost in their in-boxes or pushed to the back burner until they had free time. Attendees had the time, in that moment, to collect their thoughts and answer our questions.
2. We made a personal connection with our target audience. By explaining to them how important their input was to our study, distributing our survey in person, and making ourselves available for questions, we overcame the often impersonal nature of a web-based survey.

We recognize that this approach is more the exception than the rule. With decreased budgets and increased workloads, paper surveys rarely make sense. But in certain circumstances, they can be a powerful tool for involving stakeholders.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators

Evaluation · response rate · survey

My name is Susan Kistler, AEA’s Executive Director, and I contribute each Saturday’s aea365 post. I love finding ways to make data understandable and useful. My very first aea365 post was on Data Visualization and I gave a presentation at the 2010 AEA/CDC Summer Evaluation Institute on the Democratization of Data Inquiry (the handout, with links to example tools, may be downloaded here).

Today, I want to take a different tack and think about the intersection of art, data, and representation, being careful not to imply that beautiful graphs are not art (Tufte immediately comes to mind). Yet I hope to move beyond the printed page or the webpage through providing two examples from recent museum exhibits.

Lessons Learned: In August of 2009, I attended Roman Ondak’s “Measuring the Universe” at the Museum of Modern Art in New York. The performance exhibition consisted of having docents mark the name, height, and date of people entering a large white walled room, with each exhibit goer standing against the wall. The result was a compelling representation of participatory research in which each person contributed a data point to the resulting artwork which built over the course of a couple of months to depict the distribution of heights of New York city museum goers. My two pictures above do not do the exhibition justice – see the MoMA exhibit online for more information. I found the piece to be powerful as was the eagerness with which museum-goers, from children to seniors, wanted to participate and be recognized via measurement and recording.

Lessons Learned: In July of 2010, I visited The Tech Museum in San Jose. They had a small exhibit full of internally lit globes. Upon each globe, someone had painstakingly painted or brushed colors, added text and numbers, and illustrated a range of demographic data – the glowing yellow one above shows percent of the world’s energy use by region and the browner one is life expectancy. Again, my flat pictures do not do justice to the beauty and intrigue of the globes around which visitors from many countries walked or ducked or stood on tip toe to locate their selves within the broader context. Like the MoMA exhibit, the globes created a compelling expression of data in a way with which users wanted to interact and learn.

My question is to aea365 readers to ask if you have examples to share of data representation that bridged the gap between tables and graphs and artistic expression. What have you done? What has worked? Share in the comments section of this post (click back to the post online if you are receiving this via email or RSS), or send me an email at susan@eval.org and perhaps we can develop an exhibit space!

No tags