But I have it printed.
Well, good morning.
Thank you to the people who have stayed.
I know it's very late.
It's lunchtime.
I hope this is interesting for you and I hope
it can teach you something that you may not know.
So, I'm Ana Paula Migani.
I work as an account manager for
research intelligence matters for CIVAL.
So, let's start with the presentation.
First of all, I'd like to give a warm welcome
to everyone here and also I want to say thanks
to the organizers and the UNED for their organization.
In my intervention, I wanted to show the staff
of the library and the people working in the
organization that CIVAL is a valuable ally in
the analysis and the production of academic.
production.
I will be focusing on the last developments
that have been destined to show evidence of.
research in the real world, something that
was mentioned in a previous presentation.
Can you please change my slide?
So, as the agenda, I will show two reports.
One about the future of the research
evaluation, another one on real world.
Next, I'll talk about the framework for academic
evaluation that has a holistic, broad and
inclusive approach developed by Elsevier through these reports.
I will also talk about how CIVAL is integrated
in the academic evaluation framework from Elsevier in Spain.
CIVAL has implemented a new framework that
promotes a more comprehensive evaluation of integration.
And finally, I will show how CIVAL can be a
valuable ally in the evaluation of research.
according to the new ANECA criteria.
Universities and its researchers face a growing
pressure to prove the impact of their investigation
in society and their return on investment that the funders are hoping.
However, measuring this impact is complicated due
to the fact that it's diverse, multidimensional
and it can take decades to be materialized.
Therefore, formulating the evaluation of research
in order to reflect in a more precise way
its contribution towards society represents
a significant challenge towards universities.
In the last decade, the framework for
academic evaluation started in 2013.
It started with San Francisco evaluation, the Dora.
It started a more nuanced and intentional
evaluation in several countries such as United
Kingdom, Australia and Italy did national
evaluations that incorporated in the agreement.
Later on, we have had initiatives such as
the Leiden Manifesto, the metric type report,
the reform on the evaluation of research and
the coalition towards the advancement of the
evaluation investigation in research,
COADA, that has more than 700 signatories.
Before summer, today we have seen it's more than 800.
These efforts have promoted a more equitable,
effective and inclusive evaluation on research.
At the end of 2023, Elsevier carried out a
global survey among the academic community
in order to understand the need of changing
the evaluation of research and the obstacles that stop that.
The report that was resultant was called "Back to Earth".
The research showed a clear global trend towards
holistic evaluation, a growing public expectations
that research benefits society, a general
agreement in which there is an impact in real
world and that must be considered in the research.
We identified six obstacles that make it
harder for an effective evaluation on research.
The most mentioned one was the lack of
common frameworks and methodologies.
To understand how change should be, Elsevier
organized roundtables with academic leaders and government leaders.
The debates were summed up in a second
report called "The Future of Evaluation".
The main findings were that the evaluation
is a priority, that the main approach is the
evaluation on an institutional level, including societal impact.
We have a strong desire for change.
Finding the right balance between research and education will be key.
We require a holistic approach.
Achieving change will not be easy.
We need a cultural change.
The qualitative evaluation and the peer review
are essential in order to evaluate the bigger impact.
We need qualitative measurements in order
to measure the broader impact and AI
will play a key role in this process.
The results helped us develop an academic evaluation framework.
This framework seeks to make it easier for a
broader, more inclusive and holistic evaluation
in five key areas identified by academic leaders.
First of all, resources, then education, the
knowledge creation process, outcomes
created and its effects and impact.
Once the development of the first version of
the academic evaluation framework was done,
we made that go through the academic leaders'
review and we perfected that with the feedback we received.
The reviewed version highlights how the education
and research contributed jointly to that impact.
How do we implement this in practice?
Although we don't have an absolute model or
solution to measure real societal impact, we do have some solutions.
We use indirect and indicator measures that give
us a general idea or signal about a possible impact.
For example, one of our indicators can link
patents with the result of research from academic
essays, and some others can establish links
between documentation articles and political articles.
Now I'll explain how Saibel can help you
in a holistic evaluation on research inside
the academic evaluation framework.
Saibel is a solution for the evaluation of the performance in research.
It allows to visualize the performance of
an institution, compare it with its peers,
identify and analyze trends that are emerging in the field of research.
We have seen that the framework is structured in five pillars.
Each one of them includes several underlying elements.
The main functionalities for Saibel allow to
evaluate different levels for the sustainability,
quantity, excellence and collaboration.
Additionally, the additional functionalities
of Saibel help with human capital funding,
open science and multi-interdisciplinarity and knowledge exchange.
The new data and impact analysis from Saibel
are showing the effect of research in the
real world, showing different impact categories,
cultural, economic, environmental, health,
political, societal, technological impact.
Now I'll explain the application of Saibel in the impact pillar.
First of all, we tackle the different types
of data that we can have available in Saibel.
The nucleus of Saibel is founded on the Scopus outcomes.
Besides the bibliometric data from Scopus, the
Saibel impact analysis incorporates
information about policies and patents.
Currently, we are integrating data from quotes
in the media that offer a higher precision
in comparison to the conventional versions.
By using the compiled data in Metadase, a
database from Elsevier, Saibel allows to analyze.
the publications according to the SDGs categories
that show the research showing those goals.
Next, we are seeing how we can use the data
from policies, patents and the media and the
SDGs in order to have information on the impact pillar.
The data from policies can be used to describe
the social, political and cultural data.
Patent data help us for the economic outcomes and impact.
Information on media citations can show societal
outcomes and impact, economic outcomes and political outcomes.
And the SDGs show us the societal, political,
environmental and health outcomes and impact.
The impact analysis has become essential for
researchers and institutions because they
need to understand and show the broader impact of
their research in society in order to strengthen
institutional reputation, develop case studies,
narratives for national evaluations, optimize
Grant requests with impact evidence, improve
the recruiting, promotion and stabilization
processes and inform decisions on collaboration and training of teams.
Since we are in Spain, I would like to end
my presentation by tackling the use of Saibel
for the evaluation on research in the context
of the new ANECAS evaluation framework.
In the official state's cassette, we mention
in this table the convocation of six-year.
periods, the orange square shows different
aspects and without going into too much detail
show the advantages of Saibel in comparison with Scopus.
In Scopus, the author must review the publications
individually and register the data manually.
However, in Saibel, we offer a comprehensive
analysis of the ensemble publication by making
it easier to choose documents for different
convocations through different indicators.
That includes the combination of direct impact
indicators from the documents and the ones
from the journals that it has been published in.
Scopus can also show values from the indicators
just like the impact of citations, the percentile
of quotations, citations and public policies
documents and the links towards those documents.
However, we can't export this information in an Excel file.
However, Saibel allows the export of the values
from the indications both for documents and
for journals and it offers documentation from
citations for different documents and citations and the media.
The last ones in a close future.
In Scopus, we can see the values from the
journals just like SNP, Simago Journal Rank
and CiteScore and Source Normalized Impact Per Paper.
We include these journals and percentiles
for each one of their indicators for each document.
It also re-strengthens the narrative of the
identified documents just like the authors'
strengths in a global context, just like using
justification just like SDGs and other relevant topics.
Lastly, it also makes it possible for the
analysis of the contributions of non-academic
groups that enriches the CV narrative
that the normal CV narrative even more.
And well, I had other slides but well, the
version that I see here is not the last one that I sent.
Therefore, I'm going to stop here.
I wanted to give you some direct examples
of CIVIL application but they have not been included.
So, that's it.
I just wanted to thank you for your time, for
your participation and I'm at your disposal.
I'll also be here over the day and tomorrow
as well to give you some practical examples so you can see them.
I can do that personally then.
Thank you.