AEA365 | A Tip-a-Day by and for Evaluators

Hi, I am Laura Gagliardone. For about seven years I have collaborated with multilateral, bilateral and non-profit organizations geographically targeted to Africa, South Asia and the Middle East and focused in the following areas: research, communication, knowledge management, monitoring and evaluation, program planning, development and coordination.

RELEVANCE: I like thinking of evaluation as a way to improve programs and a chance to adjust activities accordingly to the lessons learned. A quality evaluation depends on the relevance of the quantitative and qualitative data collected and on using communication methods to raise stakeholders’ trust, interest, participation and contribution.

HOT TIP: Evaluation and communication are two disciplines which complement each other. For an evaluator, gathering and analyzing quality data is as important as transmitting the findings. Pay close attention to the communication flow among participants in order to create a positive setting where everybody is contributing to the success of the activity and gaining ownership and accountability.

HOT TIP: Position yourself as a networking hub of the program. Focus your interpersonal and communication skills in a way that installs trust, confidence and builds participant commitment. To minimize evaluation anxiety and to get a good overview of the context, an evaluator needs to be perceived as a community insider who becomes integrated and participates in daily activities.

LESSONS LEARNED: In one of the lowest income areas of Nairobi, Kenya, I evaluated an educational program for children. I used communication strategies to enhance the community’s participation and dialogue. For example, before flying to the country, I read about the cultural, economic and socio-political context and started interacting with people in the field using information and communication technologies. Once in the field, I began to empathize with locals by participating in their daily activities. While undertaking the evaluation I preferred face-to-face interviews especially with children; used open-ended questions and answers; and visited families in their houses, and brought little gifts and greeted them in ‘Kiswahili’. People in developing countries often live and work at a different speed than in Western societies, therefore they appreciate when they are given time. I have learned to appreciate peoples’ strong sense of community and capacity to interact and share. I have also noticed that individualism and independence are rarely exercised. Where possible, evaluators are advised to seek one-on-one interactions that elicit personal perspectives. As participatory techniques that elicit visioning and action planning by local community members, consider mapping exercises, supplying articles or short essays, writing brief notes, creating designs and even sharing recipes as analytical tools that inform the initiative. Enabling a participatory setting facilitates acceptance of the evaluation findings and increases ownership of the activities.

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

I’m Eric Abdullateef. I enable evaluation at USAID and facilitate interagency responses to complex crises in developing countries where the U.S. has strategic interests.  Reshaping public understanding about our initiative is a growing part of my portfolio.

When a public opinion poll is out of the question and a general pulse of how average citizens feel about your initiative will do, consider this rapid assessment approach (RAA) to gauge public attitudes about your initiative. RAAs are a quick and low cost way to gather data systematically in support of management information. Whether your public engagement effort is aimed at advocacy or education, it is important to start with a basic understanding of your audience’s beliefs and prejudices about your organization and one or more issues that your campaign is focused on.

Hot Tip: This RAA entails the following steps:

  1.  Isolate the keywords and phrases to pose answerable questions. Align these questions with your thematic unit of analysis and  the timeframe. You can delimit your search geographically by including or excluding place names. Use unlikely word combinations where possible, as these yield the best results.
  2.  Use simple Google operators for more efficient Google searches that isolate online content that address your issues. See Google Inside Search for further tips.
  3. Change your search terms or search within results to makes searches more efficient.
  4.  Instead of searching for a term across all pages on the web, search within a specialized field or venue. Google allows you to specify that your search results must come from a given website. For example, the query [ iraq site:nytimes.com ] will return pages about Iraq but only from nytimes.com. One of the fastest ways to get a sense of the public discourse around a topic is by searching within “grey literature” blogs and comments at Google Blogs.
  5. You’re limited to a search-string totaling about ten words.
  6. Select as many relevant blog posts and comments as time permits. Use a website only once to obtain more varied opinions.
  7. You’ll need a coding strategy. Keep it as simple as:
  • Enthusiasts
  • Interested
  • Neutral
  • Ambivalent
  • Skeptics
  • Rejectors

8. During the analysis process, look for and synthesize comments that signal the theory, strategy for action or evidence that your public demands in order to support your initiative. From here the evaluator should have a clearer view of public awareness of the initiative; the views of participants in the debate; group-think about what should be the criteria for success; and the extent of polarization.

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

This is Bernadette Wright. In April 2012, I made the jump to starting out as an independent consulting/sole proprietor of Wright Evaluation and Applied Research.

Lesson Learned: When I first starting exploring the idea of taking the plunge to independent consulting, it seemed like it could be a risky, scary move to make. However, I found that many resources are available to help with evaluating the decision to make this career change. These resources can help anyone considering independent consulting in evaluation feel more secure that you understand what it would take to make your business work and that you have a plan in place to make it happen. These resources can help answer questions like, Would I enjoy independent consulting? What are some good ways to find work? What does the market look like for the types of services I can provide?

Rad Resources: Below are some resources that I found to be especially helpful.

Gail V. Barrington, new book Consulting Start-up and Management: A Guide for Evaluators and Applied Researchers  is a very useful guide to all aspects of starting and managing a consulting business in evaluation and applied research. In addition, you can read a discussion led by Gail Barrington on the topic of best practices in evaluation on the AEA Thought Leaders forum and visit her website and blog .

Judah J. Viola’s 2006 article “I want to be an independent consultant: Considerations before taking the plunge,” in New Directions in Evaluation, while focused on recent graduates, provides helpful advice for anyone considering taking the plunge to independent consulting.

The online discussion forum for AEA’s Independent Consulting topical interest group is a rich source of information on things to consider in deciding to be an independent evaluator, how to find work, and tips for success. Also, AEA365 includes many posts related to this interest group.

The U.S. Small Business Administration (SBA) website on Guide to Starting and Managing a Business provides many resources on starting and managing a business, including things to consider in thinking about starting, information on where to find a mentor or counselor, and guidelines for writing a business plan.

Talking with friends, family, and co-workers can be a great way to learn from other people’s experiences and get other perspectives to consider. I was surprised to find out how many people I know have experience with independent consulting.

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Susan Jenkins.  I am a Social Science Analyst with the U.S. Administration on Aging  and on the board of the Eastern Evaluation Research Society.  I have found technical advisory groups (TAGs) invaluable for successful evaluation implementation.  A technical advisory group should be made up of relevant subject matter experts who can provide technical advice regarding the methodology, implementation and dissemination of findings of your evaluation project.

Hot Tip: A TAG can keep your evaluation on track.

  • Make sure to get input at key decision points in the evaluation and for important deliverables.
  • Develop a schedule and process for gathering TAG input. While much of the feedback can be gathered by phone, webinar and e-mail, if possible, have at least one face-to-face meeting.  I have found that such meetings set the tone for group interaction and encourage more participant commitment.
  • Request both structured and unstructured feedback. While there will be specific documents you want reviewed or approaches that you want comments on, make sure to ask some open ended questions to get feedback in areas that you may not have realized could use improvement.  I recently had to redesign an evaluation based on insights from a TAG. Specifically, the TAG pointed out that the time frame we envisioned was at odds with the grant period for the program. If we had stuck to focusing on how to make the existing methodology better most of the key informants would have moved on by the time we would have wanted to talk with them.

Lesson learned: TAGs can be helpful in a range of areas including:

  • Ensuring that you understand the fundamentals of the program from people in the field, researchers, funders and policy makers.  Make sure to include a range of people with practical knowledge and the time to devote to the TAG rather than just “big names”.
  • Building buy-in for the evaluation. When asking for data I make sure to let respondents know that the tools and approach were developed (at least in part) by respected experts and/or practitioners in their field.  I also use TAG feedback to shape stakeholders’ expectations and, when needed, to soften the blow of negative evaluation findings.
  • Supporting dissemination of findings. A well chosen TAG will have contacts and real-world experience to guide the targeting of reports/products and their dissemination.

 Rad Resources:

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hi, I’m Ann Emery. All this week, you’ll hear from members of the Washington Evaluators about the diversity of evaluation opportunities that are available in DC. Evaluation is booming in DC! This is the best city in the world for evaluators because we can choose to specialize in one area or try out different settings, approaches, and content areas throughout our careers.

Hot Tip: The only thing hotter than our humidity is our evaluation scene. Evaluators can choose to work in government agencies, non-profits, foundations, consulting firms, public schools, charter schools, or universities (like American, Catholic, UDC, GallaudetGeorge Mason, George Washington, Georgetown, Howard, and the University of Maryland, just to name a few…). Would you rather be an evaluator in the suburbs? You can also work for Virginia or Maryland’s local government or award-winning school districts.

Lesson Learned: Considering moving to DC? Don’t be alarmed by our bad traffic – we’ve got the cleanest metro system in the world, three airports, AmtrakVRE, and MARC trains, pedicabs, and some of the best bicycling trails and bike commuting in the nation. And we’ll have streetcars next year! Still running late to work? You can always blame it on the presidential motorcade

Hot Tip: Need a break from evaluating programs? On the weekends, DC evaluators can put their skills to use by evaluating the Cherry Blossoms, Redskins, food trucks, farmers markets, museums, nightlife, or even Michele Obama’s fashion choices. Still need something to do? Don’t worry, Washingtonians can talk about politics for hours!

Rad Resource: Visiting DC for a few days? Connect with the Washington Evaluators, Eastern Evaluation Research Society, or the nearby Baltimore Area Evaluators. You can mingle with evaluators at one of our happy hours or attend a brown bag while you’re in town. We welcome visitors to the Washington Evaluators monthly board meetings, and with meetings at hip restaurants in Chinatown, you can’t go wrong.

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Happy Saturday! I am Susan Kistler, the American Evaluation Association’s Executive Director and aea365 Saturday contributor. This post is Part II to the one I shared on April 28 on Inspiration and Education for Data Visualization, focusing today on two great curations of DataViz tools and tutorials. Curations are sets of items that are purposefully not comprehensive, but rather selected because the author, or curator, believes that they are of high quality.

Rad Resource Curation of Data Visualization Tools at datavisualization.ch: This newly public curation is from the team at Interactive Things, a design and tech studio in Zurich (see a portfolio of their work here). It shares approximately 40 (free) data visualization tools, including a couple that I hadn’t heard about before but can’t wait to try out. Each is presented with a small snapshot of example output.

Hot Tip: Roll over an example box and in the upper right hand corner click the small “+” – this will open up further information and site links.

Hotter Tip: In the upper right hand corner of the datavisualizatoin.ch site you’ll see Code? With a check box or x beside it. This is subtle in somewhat annoying gray on black. Select the checkbox for “I am happy to write some code on my own” and the x for “I don’t want to bother even looking at programming code.” If, like me, your programming skills are iffy, using this filter saves a great deal of time.

Clipped from: selection.datavisualization.ch (share this clip)

Rad Resource Curation of Data Visualizaton Tool Tutorials at http://compulsivedata.com/visualization-tutorials/: This set of tuturials is curated by Len de Groot who works as a Digital Media Trainer for the Knight Digital Media Center at UC Berkeley Graduate School of Journalism. While definitely not comprehensive, it can help you to fill in some of the gaps in your knowledge base, and even learn the skills needed to tackle some of the more complicated recommendations from today’s first resource recommendation. One notable challenge with this set though is that you may need to try out a number of tutorials to get a feel for the level of assumed pre-requisite knowledge.

Clipped from: compulsivedata.com (share this clip)

The above content reflects my own opinions and not necessarily those of the American Evaluation Association.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi, I’m Jade Caines. I have taught prekindergarten through college grades for over 10 years and have also worked on numerous evaluation studies in the southern, western, and eastern parts of the U.S. I recently received a Ph.D. in Educational Research, Measurement, and Evaluation. Currently, I am a postdoctoral research fellow at University of Pennsylvania, where my research area includes the validity of scales and instruments used within education evaluations.

Lessons Learned:

I have been in many situations where people wanted to know whether or not something “worked.” Folks would slap together some questions, put it on Survey Monkey, and get some people to respond. But since my involvement in the Educational Measurement field, I have learned so much about survey design. The most critical lesson I’ve learned in creating surveys used within education evaluations is the importance of defining exactly what it is that is being measured. Often I’ve had conversations with clients where the “what” is taken for granted. I have learned to spend significant time probing, asking pointed questions about what they really want to measure. Eventually, clients realize that they may not be so sure about the one “thing” they want to measure. Questions, Questions, and more Questions have helped in defining a construct (that “thing”) and then operationalizing it (creating situations where that “thing” is represented) for survey design. A construct can be defined as the theoretical object of our interest in the respondent. For example, character in students may be a construct that evaluators may want to measure as a part of a character evaluation grant.

Hot Tips:

  • Name the “thing” you want to measure (e.g. perseverance in the classroom).
  • Create a list of situations/experiences where that “thing” is represented (e.g. a student resubmits a writing assignment three times for a better grade). This would be a list of indicators, or evidence that a certain “thing” exists.
  • Create a list of situations/experiences where the opposite of that “thing” is represented (e.g. a student chooses not to submit an assignment, despite multiple deadline extensions from the teacher).
  • Then decide how to represent these situations that span a continuum of that “thing” on a measurement tool (e.g. a survey).

Cool Trick:

Draw a vertical line where the top of the line represents a high amount of that “thing” and the bottom represents a low amount. Try to create a survey that has several questions in the high, medium, and low sections of that line.

Rad Resource:

Constructing Measures: An Item Response Modeling Approach by Mark Wilson (2005). The first 2 chapters are the most relevant.

Hot Tip: Take a minute and thank a teacher this week!

The American Evaluation Association is celebrating Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · ·

I am Veronica Smith, principal of data2insight, an evaluation and research firm specializing in science, technology, engineering and math (STEM) program evaluation.

We worked with Washington Global Health Alliance (WGHA), a consortium of  research universities and non-profit organizations in Washington state, on the evaluation of an interdisciplinary curriculum program called Ambassadors. WGHA Ambassadors (WGHAA) aims to give high school students a well-rounded perspective of global health and how global health challenges need to be met by creative new ideas. A key part of this pilot program was the development of 11th grade algebra, chemistry and United States history curricula organized around global health diseases. The curricula was designed by high school teachers using a framework called Understanding By Design led by Laughing Crow Curriculum experts.

As an external  evaluator, I  provided teacher professional development on formative and summative learning assessment because one key evaluation question was “What have students learned from WGHAA lessons?” In partnership with the curriculum design expert, we facilitated  workshops over the course of a year that provided the framework for assessment for learning (formative) and assessment of learning (summative). The results of this hybrid professional development-curriculum design-evaluation effort included:

  • A pretest-posttest and scoring guide used to measure  learning gains in algebra, chemistry and U.S. history for both teachers and program evaluators.
  • 42 formative lesson assessments with scoring guides, some of which were used as common assessments across same-subject classrooms for evaluation of learning gains from core lessons
  • Teachers reported a better sense of what to assess, how and when to assess, and felt  they could develop better learning assessments

Rad Resources:

  • Thanks to funding by The Bill and Melinda Gates Foundation and WGHA, this curriculum is free from the WGHAA website.
  • We used Teacher-Made Assessments, by Christopher R. Gareis and Leslie W. Grant, to guide development of the pretest-posttest. The “How Do I Create a Good Test?” chapter provides a systematic process for test development that aligns with the use of Bloom’s taxonomy of cognitive demand.
  • For in-service sessions with teachers, I used formative assessment classroom techniques (FACTs) from Page Keeley’s book, Science Formative Assessments. This book provides teachers with research-based guidance, suggestions and techniques for using formative assessment to improve teaching and learning in K-12 science classrooms, and can be used for other disciplines as well.

Lesson Learned: Asking teachers at the beginning what topics would be most useful for them resulted in workshops on classroom formative assessment, student self-assessment and using assessment data. Developing training based on teacher preference helped ensure participant engagement and topic relevance.

For a copy of the WGHAA evaluation report, email your request to veronicasmith@data2insight.com.

Hot Tip: Take a minute and thank a teacher this week!

The American Evaluation Association is celebrating Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hi! My name is Tiffany Berry and I’m a research associate professor at Claremont Graduate University. My colleague, Kathryn Edwards, is an educational evaluator from the Los Angeles County Office of Education (LACOE).

I recently attended a training hosted by LACOE on the assessment systems that will replace our state standardized tests starting in 2014-2015. As an educational evaluator, I found this training invaluable given that (1) we rely on state testing as a key achievement measure in many evaluations; (2) formative and summative assessments are our bread and butter; (3) we need to be mindful of interpreting and using state assessment data during the transition years; and (4) we may be called upon to help educators understand, use, and validate the new assessments as well as plan for their impending implementation.

Rad Resource: Latest Information about the Common Core Standards

The Common Core Initiative is a state-led effort launched by the National Governors Association and the Council of Chief State School Officers (CCSSO). These K-12 English Language Arts and mathematics standards were developed in collaboration with teachers, school administrators, and experts, to provide a clear and consistent framework to prepare students for college and careers. The final standards, released in June 2010, have been adopted by forty-five of the fifty states. States are in the process of developing implementation plans to facilitate transition to the Common Core. Please visit this website for more information: http://www.corestandards.org

Rad Resource: Assessment Systems Being Developed to Align with Common Core

Two multi-state consortia, the Partnership for Assessment of Readiness for College and Career (PARCC) and SMARTER Balanced Assessment System received Race To The Top funding to build next generation assessment systems to measure the full range of the Common Core State Standards. The consortia will use on-line systems to test students in grades 3 through 12 using interim and summative assessments. These innovative systems will deliver a variety of item types including selected response, constructed response and performance tasks. Additionally, the consortia will provide resources and training for educators.

Hot Tip: This post is intended as a call to action for educational evaluators serving PreK-12. Knowing when these assessments come on-line, what constructs they measure, how they measure them, and how these assessments intend to inform student learning will position educational evaluators to facilitate important conversations around how best to use research, evaluation, and assessment to support educational institutions in the 21st century.

Hot Tip: Take a minute and thank a teacher this week!

The American Evaluation Association is celebrating Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · ·

I am Deborah Mattila, Research and Evaluation Director at the Improve Group. Over the last several years I have had the pleasure of evaluating many school-based initiatives that explicitly and implicitly address development and application of 21st Century Skills, a set of learning and innovation skills built around the 4 Cs of Critical Thinking, Communication, Collaboration, and Creativity.

Rad Resources:

  • The Partnership for 21st Century Skills (P21, the national organization advocating for 21st Century readiness) has an interactive road map for 21st Century skills-related information, resources and tools. Here you can check out details behind all the skills areas, and their connections with standards-based learning, instructional practice and learning environments.
  • Intel® Teach Elements are free professional development courses that can help educators, program staff, and evaluators understand different aspects of a 21st Century Classroom (focused on digital learning), develop and use authentic assessments, and examine what student-led data-focused critical thinking looks like.

Hot Tips:

  • A key tenet of 21st Century classrooms is authentic assessment of student learning and achievement. Authentic assessments, which are developed to closely match the expected learning goals and desired skills, can be a great source of documentation for your evaluation; they can give you a full picture of what learning skills students are developing.
  • Many of the skills emphasized in 21st century learning—such as critical thinking or creativity —feel to children like a natural part of who they are, not unique, stand-alone skills. Mixed methods in evaluation – surveys, classroom observations, review of authentic assessments—give a broader view of how 21st Century Skills manifest in each student.
  • 21st Century Skills are not limited to either elementary or secondary grade levels, or to any one subject area. This is important because you can look for student, teacher and learning environment outcomes related to the 21st Century skills, even when they are not an explicit goal of your program or initiative.

Lessons Learned:

  • Teaching staff may see 21st Century Skills as one more demand on their already burdened teaching practice. It may also feel like the pedagogical “flavor of the month” if their administration responds frequently to new content or teaching strategies. In particular, as the Common Core Standards is pushing to the forefront of K-12 education, the 21st Century Skills may be left behind. I have found that framing 21st Century Skills as how kids learn, rather than what kids learn, helps focus the conversation on what we will and will not measure.

I love connecting and sharing ideas with other evaluators – connect with me on Twitter to have a conversation about this or other #eval topics!

Hot Tip: Take a minute and thank a teacher this week!

The American Evaluation Association is celebrating Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Older posts >>

Archives

To top