The Sinaes policy in the Federal Institute: the regulatory presence of the State in the consolidation of a model of Higher Education in a non-university institution[1]
Fabíula Tatiane Pires[2]
https://orcid.org/0000-0002-9649-3596
Suzana dos Santos Gomes[3]
http://orcid.org/0000-0002-8660-1741
Abstract
This article analyzes how the policy of the National System of Evaluation of Higher Education (Sinaes) has been put into practice in a Federal Institute (FI) of Minas Gerais. From a bibliographic research, in the light of Stephen J. Ball's policy cycle, it presents the influences of international organizations on Sinaes and on the creation of FI, conceived as an alternative to offer low-cost higher education courses. The empirical data, obtained through the application of a questionnaire and semi-structured interviews with teachers, were explored based on Laurence Bardin's content analysis. The results show the predominance of a Sinaes with a more regulatory character, which, in turn, contributes to the consolidation of the FI as a model of non-university Higher Education.
Keywords: Higher Education. International Organizations. Federal Institute.
A política do Sinaes no Instituto Federal: a presença reguladora do Estado na consolidação de um modelo de Educação Superior em instituição não universitária
Resumo
Este artigo analisa como a política do Sistema Nacional de Avaliação da Educação Superior (Sinaes) tem sido colocada em prática em um Instituto Federal (IF) mineiro. A partir de uma pesquisa bibliográfica, à luz do ciclo de políticas de Stephen J. Ball, apresenta as influências de organismos internacionais no Sinaes e na criação dos IF, concebidos como uma alternativa de oferta de cursos superiores de baixo custo. Os dados empíricos, obtidos por meio da aplicação de questionário e entrevistas semiestruturadas com docentes, foram explorados com base na análise de conteúdo de Laurence Bardin. Os resultados evidenciam a predominância de um Sinaes com caráter mais regulador, o que, por sua vez, contribui para consolidar o IF como modelo de Educação Superior não universitária.
Palavras-chave: Educação Superior. Organismos Internacionais. Instituto Federal.
Introduction
This article is part of a doctoral research project that focuses on higher education at a Federal Institute of Education, Science, and Technology (FI) in the state of Minas Gerais, and more specifically on the courses offered at one of its campuses. For the purposes of evaluation, regulation, and supervision, the FIs are equivalent to federal universities, as stated in the law that created them (Brazil, 2008). However, although they are similar to universities in the provision of higher education courses, they retain elements of an identity under construction, among which the following stand out: the verticalized provision of education, in which both basic and higher education courses are offered in the same physical space and with the same group of teachers and technical-administrative staff; the linking of courses to local productive arrangements with a view to the social and economic development of the regions in which they are located; and, finally, the development of research and extension activities, as well as the curriculum of courses with a technological bias.
Although the Federal Institutes (FI) are equivalent to federal universities for the purposes of evaluating, regulating, and supervising the institution and its programs, as expressed in Law No. 11.892 of 2008 (Brazil, 2008), they are not universities. Otranto (2010), after analyzing World Bank documents, concludes that at the time of the creation of FIs, the intentional effort of the Bank was to promote the diversification of higher education institutions, to which FIs responded as an alternative institution, "since research universities had been identified by the World Bank since the 1990s as too expensive for developing countries" (Otranto, 2012, p. 202). In this sense, this article characterizes the FI as a non-university institution, considering the influences exerted by the World Bank on the policy of its creation, in addition to its vertical structure of teaching, ranging from initial and continuing training courses to basic and higher education.
It was important to know more about the courses offered by the FIs: 1) because they are a new place for training teachers who will work in basic education; 2) because of the legal requirement, Law 11.892 of 2008 (Brasil, 2008), which stipulates that 20% of their vacancies must be reserved for courses of this type; 3) and because these institutions, which have traditionally offered professional and technological education, have little or no experience in teacher training courses. Given the equalization of institutions of different types in a system of evaluation, regulation, and supervision of higher education, and considering the international influences on this policy, the question arises: how has the policy of the National Higher Education Evaluation System (Sinaes) been implemented in the FIs?
This article is divided into sections: the first explains the design of the research in its methodological aspects; the second presents the elements of the policy cycle by Stephen J. Ball and his collaborators - a reference point for the research. Ball and his collaborators - the theoretical-methodological framework adopted to understand the Sinaes policy and the creation of FIs; the third section, based on bibliographical research, discusses these policies in the light of international influences, based on empirical data obtained through the application of a questionnaire and semi-structured interviews with teachers working in three degree courses at an FI in Minas Gerais, interpreted from the perspective of Bardin's (2016) content analysis; the fourth section highlights the context in which Sinaes is practiced in the institution; and, finally, the concluding reflections summarize the discussions, showing how the influence of international bodies has strengthened the regulatory presence of the state in FIs, contributing to the consolidation of a model of higher education that goes beyond university institutions.
Methodological path
To answer the question at the heart of this article - how has the policy of the National Higher Education Assessment System (Sinaes) been implemented in the Federal Institutes (FI)? We first sought to understand the policy cycle developed by the English researcher Stephen Ball and his collaborators. This theoretical-analytical framework allows us to understand public policies in motion, highlighting disputes, resistance, and re-readings.
At the same time, a bibliographical study of the Sinaes policy and the creation of the FIs was carried out, with the aim of explaining, in the light of the policy cycle, the contexts of influence, text production and practice of policies in Brazilian higher education.
Finally, to understand the context in which the Sinaes policy is practiced in an FI, the following criteria were adopted: 1) an FI in Minas Gerais, since the researchers reside in this state and the study did not receive funding; 2) a campus with a tradition of offering Professional and Technological Education (PTE), since this modality is the main identity of these institutions; and 3) a campus that offered the largest number of degrees, including those provided for in the law that created the FIs.
Once the campus was selected, the teachers of the three majors - Chemistry, Biology, and Physical Education - were invited to participate in the survey by answering a digital questionnaire, since at the time of the survey, Brazil was facing the COVID-19 pandemic and social distancing was the main public health and safety measure adopted.
From a group of 50 teachers, 25 responded to the questionnaire, expressing interest or disinterest in continuing to participate in the research. After analyzing the questionnaire, the profile of the teachers for the semi-structured interview phase was defined: 1) one teacher from each program; 2) one coordinator or vice-coordinator from each program; 3) one teacher with experience in a permanent evaluation commission or equivalent body; and 4) one teacher with a managerial position. Based on these criteria, eight teachers were invited, five female and three male.
The data collected were examined using content analysis, as suggested by Bardin (2016). First, a floating reading of the transcribed corpus of interviews was carried out. Based on the defined objectives, primary categories were indicated, followed by intermediate categories. The final categories were the result of an interpretive effort of the relationship between the parts and the whole, in dialogue with the theoretical framework adopted.
Contributions of the theoretical-analytical method of the policy cycle to understanding educational policies
To understand the policies of Sinaes and the creation of FIs (Brazil, 2004, 2008), we decided to adopt the theoretical-analytical method developed by Bowe, Ball and Gold in the 1990s, called the policy cycle. In Brazil, the policy cycle was popularized in the work of Jefferson Mainardes, a researcher at the Universidade Estadual de Ponta Grossa. For Mainardes (2006), this approach allows policies to be understood in a dynamic and flexible way, that is, in constant movement and transformation. In this approach, the different moments of proposing the agenda, drafting the text, and putting it into practice are interrelated and can influence each other.
In the early 1990s, Bowe, Ball, and Gold (1992) characterized and distinguished three main contexts of the policy cycle: influence, production of the policy text, and practice. After revising the method, Ball (1994) added two more contexts to the cycle: that of results or effects and that of policy strategy. Years later, in an interview with Mainardes and Marcondes (2009), Ball considered the possibility of including the context of results or effects in that of practice and that of political strategy in that of influence, maintaining the integrated action between them.
The context of influence is characterized by discourses constructed in the midst of clashes and ideological disputes. Different actors position themselves in the political arena, defending different interests. The discourses with the largest number of supporters will be the strongest and most representative, while others may be resisted, silenced, modified, or even extinguished, only to be taken up again at another time. According to Bowe, Ball, and Gold (1992), policy can be influenced by commissions, organizations, and committees at different levels: global, international, national, and local. Thus, educational funding banks, project promoters, and teacher and researcher associations, among others, have the potential to influence educational policies. However, this influence is not evenly distributed.
Global and international influence, according to Mainardes (2006, p. 51), can occur through the "flow of ideas through political and social networks"; through political and academic publications; or through the funding of "solutions" to educational problems proposed by multilateral organizations. Influences are always interpreted in the light of national and local scenarios. Including the context of political strategy in that of influence, as suggested by Ball (apud Mainardes; Marcondes, 2009), makes it possible to identify strategies for overcoming clashes or anticipating problems in debates.
In turn, the context of the production of the policy text, according to Bowe, Ball, and Gold (1992), maintains a close relationship with that of influence, since the discourses legitimized during agenda setting are represented in the production of the policy text. This text manifests itself in official regulations, normative instructions, curriculum proposals, and other written materials. In addition to the main policy text, other documents may be produced to clarify it, since "policy texts are not necessarily clear and coherent" and are "the result of struggles" (Bowe; Ball; Gold, 1992, p. 21). The lack of coherence between texts of the same policy can lead to contradictions, and for this reason, they must be read in the light of the context and other texts that may even contradict them. For Mainardes (2006), the policy text can be of two types: prescriptive, in which the reader attributes meaning to what he reads based on his repertoire; and "writable," in which the reader is a co-author through creative and active interpretation. In this sense, "texts have a clear link to the particular contexts in which they are produced and used" (Mainardes, 2006, p. 50).
Finally, in the context of practice, according to Bowe, Ball, and Gold (1992), the centrality lies in recognizing that policy is not simply implemented as proposed, but is subjectively and contextually interpreted, and can even be recreated, making it subject to dispute. Policies are then translated into a sphere of complexity (Mainardes; Marcondes, 2009). For Ball (1994), policies, through their texts, intervene in practice by defining goals and objectives that can change or maintain a given reality. Policies themselves can be modified by reality. By considering, as Ball did in an interview with Mainardes and Marcondes (2009), the inclusion of the context of results or effects in the context of practice, it is recognized that the practice of a policy has the potential to have an impact on situations of injustice and inequality. According to Mainardes (2006), the effects of a policy can produce such significant changes that they end up changing the original policy.
For the discussions in this article, the focus is on the context of influence, especially that of international organizations, on both the Sinaes policy and the policy that created the FIs, but also on the context of the practice of these policies in an FI in Minas Gerais.
Sinaes policies and the creation of the FIs: analysis of the context of influence based on bibliographic production
Sinaes is the longest-running higher education evaluation policy in Brazilian history. Both the proposal and the practice of this system are marked by conflicts of interest between the state, multilateral organizations and organized civil society, including teachers and researchers. With the expansion of higher education, the evaluation of the quality of education offered by higher education institutions (HEIs) has become an imperative.
The restructuring of the Brazilian state, which began at the end of the 20th century, was guided by a discourse of modernization and debureaucratization of public administration. For Zatti (2017), since the 1970s, Brazilian public policymaking, in its discourse and texts, began to adopt language and devices typical of the market, emphasizing the role of evaluation, accountability, and performance measurement. According to the author, at the international level, the idea of a link between education, knowledge, and economic and social development was consolidated, including recommendations to adopt elements of business management in the management of education.
According to Zatti (2017), several international organizations have also had a strong influence on education reform in Latin America, especially in Brazil. These include: 1) the Economic Commission for Latin America (ECLAC), which in 1990 recommended the democratization of education to make countries more competitive, suggesting that education systems adapt the supply of knowledge and skills to the needs of the productive sector; 2) also in 1990, the World Conference on Education for All, financed by the World Bank and the United Nations Educational, Scientific and Cultural Organization (Unesco), advocated expanding access to basic education, with the definition of goals and policies that could be evaluated to establish a minimum standard of quality and measure learning outcomes; 3) in 1992, ECLAC, in partnership with Unesco, developed guidelines that systematically linked education, knowledge and economic development.
Specifically regarding Sinaes, Daros Júnior (2014), in examining the influences of Unesco, concluded that although there were no formal agreements between Unesco and Brazil for the creation of Sinaes, the organization indirectly influenced its creation through the members of the Special Evaluation Commission (SEC), who maintained some connection with Unesco. According to Daros Júnior, their ideas were convergent, linking the education of citizens to economic and social development, defending equal access and the inclusion of minorities, giving the state responsibility for evaluation, regulation, and control, and accepting the opening of higher education to private initiative. In addition, they advocated external evaluation of courses and institutions with internal evaluation, respect for the diversity of higher education institutions, and the implementation of non-university models. However, the proposals differed in terms of the definition of quality in higher education: for UNESCO, quality was linked to international standards, including the ability to exchange knowledge, network interaction, mobility of teachers and students, as well as the proposal of international research projects, while for the members of the SEC, the benchmark was centered on public universities.
The international influence can also be seen in the Law on Educational Guidelines and Bases (EGB), Law No. 9.394 (Brazil, 1996), which incorporated part of the international recommendations, especially about higher education. The LDB linked evaluation processes to the authorization and recognition of courses, as well as to the accreditation and de-accreditation of higher education institutions (Article 46). In this way, evaluation has become the main instrument of accountability and has even promoted competitiveness among higher education institutions. It is worth noting that accountability is necessary for higher education institutions to reaffirm their commitment to society. However, when it is guided exclusively by market interests and intensifies competitiveness between universities, evaluation turns into rankings, bonuses, and/or punishments for performance, which, according to Sousa (2008), trade unions criticize and oppose.
In 2004, the publication of Sinaes showed the strength of international influences. According to Maués (2007), the final Sinaes document did not fully reflect the concept of evaluation defended by the CEA, the commission in charge of discussing and presenting the proposal for this policy, suggesting that these influences met resistance during the debates on its proposal:
The concept proposed in the CAA report, according to which the concept of evaluation should include inclusion and participation, has not been translated into specific legislation, which is rather identified with a concept of evaluation that represents a more technical and instrumental aspect, i.e., one that involves the collection of data and information, but does not subject it to a process of reflection and questioning. On the contrary, this type of evaluation only serves to hold institutions, sectors, and people accountable (Maués, 2007, p. 13).
This excerpt shows the influence of international organizations in terms of a more instrumental and quantitative evaluation. The influence of these actors was also manifested in the definition of the objectives of Sinaes, including:
Improve the quality of higher education, guide the expansion of its supply, permanently increase its institutional efficiency and academic and social effectiveness, and especially promote the deepening and social commitments and responsibilities of higher education institutions [...] (Brasil, 2004, art. 1, § I).
The text of the policy emphasized the managerial aspects of effectiveness and efficiency, as well as the orientation of higher education because of rational and supposedly ideologically neutral evaluation criteria. However, as a result of controversies, the Sinaes policy (Brazil, 2004), in its Article 1, reinstated the principles defended by the CEA and the National Teachers' Association (Andes) in the 1980s, such as integration, globalism, publication of procedures, data and results, respect for institutional identities and diversities, and collective participation in the evaluation process.
Law No. 10.861 (Brazil, 2004) reaffirmed in its only paragraph, Article 2, that the results of the evaluation would be used as a basic reference for the processes of regulation and supervision of higher education, i.e., two different and complementary purposes. For Verhine (2015), the novelty of Sinaes consisted precisely in the articulation between the formative and regulatory processes - antagonistic forces that were in conflict and would be instrumentalized by the evaluation, which was intended to:
[…] produce information that is scientifically valid, reliable, and comparable over time, through processes carried out by specialists who act free from external interference that could damage the quality of the knowledge produced. Government regulation, on the other hand, seeks to comply with policies established by specific administrations and, in the case of higher education, can promote the achievement of expansion and quality goals in accordance with the interests of those in power and the population that supports them (Verhine, 2015, p. 616).
The text of Sinaes established that the process of evaluating higher education would begin, based on institutional self-assessment - a formative moment in which higher education institutions would expand their knowledge of themselves through a collective process of reflection. Subsequently, external examiners would visit the institutions, and the results would be complemented by course evaluations and the National Student Achievement Exam (Enade). However, according to Barreyro and Rothen (2014), after the enactment of the Sinaes Law, it was modified and ignored by smaller normative acts, showing that international influences continued to be active. For the authors, there was:
Two SINAES: the one proposed by the Special Evaluation Commission and the one established by law. The second only partially recovered the emancipatory principles of the first proposal, reconciling them with the regulatory vision of the Provão era. During the period of implementation, the tension between these two visions was permanent (Barreyro; Rothen, 2014, p. 66).
Barreyro and Rothen (2014) add that on-site visits by external evaluators were replaced based on the preliminary satisfactory concept before being consolidated, transforming the process into a quantitative ranking trend. Decrees and ordinances modified the initial approach to evaluation. In 2006, Decree 5.773 (Brazil, 2006), known as the Bridge Decree, regulated the regulation, supervision, and evaluation of higher education institutions and programs. For Barreyro and Rothen (2006, p. 969), the Sinaes Law presented "contradictions that need bridges" and reconciliation between different trends:
With the Bridge Decree, the law establishing SINAES is "reworked": on the one hand, it takes up the principle of separation between regulation and evaluation; on the other hand, it reinforces the vision of separation between self-evaluation and external evaluation, clarifying it that the latter is part of the process of regulation and supervision (Barreyro; Rothen, 2006, pp. 970-971).
According to the authors, the conception of Sinaes has undergone a kind of metamorphosis based on previous evaluation experiences based on different paradigms and values. For Dias Sobrinho (2008), the first four years of Sinaes were marked by a model that distanced evaluation from the exclusive compliance with bureaucratic norms and the indication of positions in rankings. However, through successive regulations, institutional evaluation lost its centrality in the process, and Enade results began to guide the process of higher education evaluation. This change indicates that:
A simple change in methodology. It is, rather, a radical change in the assessment paradigm: from the production of meaning and reflection on the values of knowledge and training, to control, selection and classification on numerical scales (Dias Sobrinho, 2008, p. 821).
Thus, Sinaes is a law that combines elements of regulation and accountability with a perspective of integration and globalization. For Gomes, Melo and Garrido (2022), the proposed adjustments, especially the recommendations of the World Bank, are leading universities towards a new model that has met with resistance from the academic community. However, the authors warn that these changes are gradually being incorporated into federal decrees, programs, and projects. For Macedo and Araújo (2022), these changes increasingly position higher education as a commodity, emptying it of its social dimension.
In the context of the influences and proposals of international organizations for higher education, in 2008 Brazil established the Federal Network for Professional, Scientific, and Technological Education, as well as the creation of FIs from pre-existing institutions in all units of the Federation (Brasil, 2008). The concept of FIs allowed not only the expansion and internalization of the provision of basic education, professional and technological education (PTE), but also higher education, since with vertical teaching, the institution offers courses at the different levels and modalities of national education.
From a neoliberal perspective, PTE has also been influenced by the desires of the state, international organizations and economic interests. According to Ferreira (2017), the global agenda conceived and financed by these organizations has been closely guided by the desires of the market, with the clear objective of producing workers capable of maintaining the accumulation of capital. The author found convergences in the proposals of the Inter-American Development Bank (IDB), the International Bank for Reconstruction and Development (IBRD), the World Bank and the Economic Commission for Latin America and the Caribbean (ECLAC). For these institutions, according to the author, PTE aims to improve professional skills, which would promote the social and economic development of countries. Using a discourse aimed at the common good, these actors blame education for the socio-economic backwardness of countries. In addition, they argue that vocational training should be adapted to market demands, which implies shorter courses so that workers can quickly enter the market, as well as opening up PTE to private initiative.
In this context of influences, the FIs emerged from the transformation and aggregation of the federal network's vocational training institutions, which had traditionally provided PTE at the secondary level, and expanded their remit to include the provision of higher education in courses in which they had little or no experience. As far as teaching is concerned, the FIs adopt a verticalization proposal, i.e., basic education and higher education courses are offered in the same training itinerary. Verticalization allows courses to be offered at a lower cost by sharing physical space and human resources. Another feature of teaching in FIs is the 20% reservation of vacancies for undergraduate courses and/or the Special Teacher Training Program (Brasil, 2008), a strategy to try to make up for the shortage of teachers, especially in the areas of science, mathematics, and PTE.
Studies such as Otranto (2013), Borborema (2013), and Silva and Bifano (2020) explain the World Bank's influence on the provision of higher education in FIs. According to Borborema (2013), the World Bank's influence was based on the discourse that the courses offered by traditional universities were expensive and did not meet the needs of the country's economic and social development. Therefore, the World Bank proposed the creation of non-university institutions, greater openness to private initiative, and the provision of shorter courses. Otranto (2013) adds that this proposal includes flexible courses with lower costs than traditional university courses.
However, the model of non-university higher education predates the creation of FIs. Borborema (2013) points out that this model began to be implemented in Brazil in the 1990s, with the Federal Centers for Technological Education (Centros Federais de Educação Tecnológica - Cefet), which began to offer higher education courses in the industrial sector. According to the author, the structure of the FIs is very similar to that of the Cefet, as both enjoy financial, administrative and pedagogical autonomy. Currently, the federal PTE network has FIs in all Brazilian states, two Cefets, in Minas Gerais and Rio de Janeiro, and the Universidade Tecnológica Federal do Paraná (UTFPR).
The main difference between Cefet and FI is that Cefet can offer secondary education independent of technical training, while FI cannot. In turn, the FIs differ from the UTFPR in that there is no obligation for the university to reserve 50% of places to offer PTE at the high school level, nor to allocate 20% of places to degree courses. These points indicate the greater autonomy of Cefet and UTFPR in relation to the FIs.
In 2003, according to Silva and Bifano (2020), the International Bank for Reconstruction and Development (IBRD) defined higher education as tertiary education, i.e., education after secondary education. In this sense, higher education would go hand in hand with that offered by polytechnic institutions, for example, and would imply the provision of "skills or training and make it possible to train more easily and quickly" (Silva; Bifano, 2020, p. 116). For the authors, the IBRD offered Brazil not only financial resources, but also an ideological package.
Although the World Bank, through the work of the IBRD, recommended replacing higher education with PTE, especially for the less privileged classes, and blamed countries that did not follow this guideline for their lack of competitiveness in the global economy, Brazil, as Silva and Bifano (2020) point out, interpreted this recommendation differently. The country created FIs and equated them with federal universities for the purpose of evaluating, regulating, and supervising institutions and their higher education courses. Thus, there was no substitution of higher education by FIs, but rather an expansion of both, with the optimization of physical spaces and human resources.
In conclusion, the bibliographic production based on the contributions of the policy cycle showed that the influence of international organizations on Brazilian higher education and PTE was materialized in the texts of the main policies for these segments. However, it was also found that these influences were interpreted in the light of the national context, with reservations and transformations.
The context in which the Sinaes policy is practiced at a federal institute in Minas Gerais
As mentioned above, the policy cycle approach, while distinguishing three main contexts in a policy, recognizes that these are integrated, dynamic parts that can influence each other. We sought to understand the context in which the Sinaes policy is practiced based on the experiences of teachers of three-degree courses at a campus of a Federal Institute (FI) in Minas Gerais.
The campus selected for the research is one of the oldest in the FI and offers the largest number of undergraduate degrees. In 2022, 50 teachers were working on degrees in Physical Education, Chemistry and Biological Sciences, according to the information in the pedagogical projects of the courses. Twenty-five of these teachers accepted and responded to the online questionnaire and, based on the criteria established in the research, eight were selected to participate in the interview phase. All the teachers interviewed are tenured, full-time, and have been working in the programs for the same period or close to the period of implementation. In addition, they all have some experience in leadership positions or participation in the Institutional Evaluation Committee (IEC), which is responsible for carrying out the institutional evaluation process. As approved by the Research Ethics Committee, all the names of the respondents are fictitious, thus guaranteeing their anonymity.
For 92% of the teachers who replied to the questionnaire, the SINAES policy, especially in the course evaluation phase, allows the institution to identify the strengths and weaknesses of its teaching staff, pedagogical organization, and infrastructure. For 96% of the teachers, the external evaluation of the courses is an opportunity to highlight the progress, achievements, and challenges of the courses. Concerning the quality of the courses, 76% of the respondents think that the evaluation provides indicators to control the quality of the courses.
These teachers know the Sinaes policy mainly from its micro context of practice, i.e., course evaluation. Enade was not mentioned as part of this policy, and institutional evaluation was only vaguely mentioned:
[...] people are not prepared, they do not do this evaluation [institutional evaluation], they are lazy, they let it go, and they see it as a trivial thing. In addition, I think we have to change our culture, you know? We need to work on this change (Prof. Aline).
Despite the challenges and the culture of evaluation, Professor Álvaro recognizes that institutional evaluation is an opportunity for self-knowledge, diagnosis, and planning:
[…] therefore, we can get feedback on our work, sometimes we think we're doing everything right and sometimes the evaluations will tell us if we're on the right track or if we need to make some adjustments (Prof. Álvaro).
Although institutional evaluation has the potential to provide moments for reflection, diagnosis and planning, the teacher's statement highlights the limitations of the evaluation culture at the FI, characterized by the absence of a deliberate action that highlights the importance of the moment and arouses the interest of teachers to participate in this process. For Magalhães and Rodrigues (2021, p. 52), Sinaes sought to strengthen the participation of the academic community in the evaluation of higher education, understanding that:
The community should be actively involved in evaluation processes, and self-evaluation processes were also valued, considering that the knowledge of those involved in university processes could contribute to their improvement.
However, this challenge remains on the campus in question. Facing this challenge is linked to a greater mobilization and awareness of the academic community, which is still confronted with the perception of Sinaes not as an emancipatory proposal, but yet another control mechanism over the institution.
According to Professor Aline, one of the factors that limit the scope of institutional evaluation is the abundance of quantitative data that reveal little about the concrete reality of the institution:
So sometimes, when I see these MEC evaluations, I get a little scared because I look at them with a coldness of numbers. [...] we, all of us, only get five and... That's it, I think we are much more than five (Prof. Aline).
Professor Simone also points out that the Sinaes policy does not consider the reality of the FIs, nor the different nature of higher education courses:
I think that in addition to the lack of a more detailed look at the contexts, for the federal network in general, there is also a lack of a more refined look at the contexts [...] The other issue is to think about the particularities of the institutes [...] so the type of productivity, the productivity index, the involvement in the production of knowledge, whether through teaching, research, or extension, is different from, for example, in the health sciences than in the human sciences. [...] So I think you have to think that the institutes are evaluated differently than the universities because the logistics of the teachers' work are different, because of this demand to be involved with different levels, so I prepare a physical education lesson for the technician with an approach that may be the same content but will have a different approach in the high school, and the same content in the degree that will have an entirely different approach (Prof. Simone).
The lack of a higher education evaluation tool that makes the different institutional realities visible has already been criticized by Moraes et al. (2013). According to the authors, SINAES "did not make any changes to the external evaluation tool to consider the specificities of the federal institutes" (Moraes et al., 2013, p. 36). In this sense, the results of the SINAES evaluations of the campuses studied are limited in terms of gathering real information about the institution and function more as a means of verifying compliance with previously established criteria.
The result of the clash between what is done in the FIs and the evaluation criteria of the SINAES policy imposes on the institution, the courses, and their teachers the need for adaptations that are sometimes little reflected or of little relevance to the courses, considering the local context in which they are inserted:
So, as I said about the evaluation tool for university courses, we have to adapt to what the universities say, but we don't have the same incentive as the universities. [...] Here we have a workload of 16 to 20 classes, so how do you manage 20 classes a week, plus research and extension? So I think this is a huge challenge in terms of assessment itself. [...] What also happens a lot is that we often change to meet this criterion and not because we think it is actually the best for the course, but because we have to meet this X criterion (Prof. Amélia).
If FIs are institutions "with such unique characteristics, it is because their place on the educational scene, one imagines, has been strategically designed to fill a role not yet filled by existing institutions" (Moraes et al. 2013, p. 39). Thus, through a drastic regulatory action, the evaluation of higher education modifies the very raison d'être of the FI, shaping it according to university standards.
In the case of verticalized teaching, where teachers work in both higher education and basic education, the workload allocated to teaching compromises other professional activities, such as research and extension. However, among the criteria evaluated by SINAES, it is precisely research and knowledge production that, according to the teachers, score highest in the evaluation of the courses:
It comes from the top down: we start with academic productivity itself, producing articles, publishing books, magazines, participating in editorials in academic journals, and then linking it to teaching and extension (Prof. Simone).
The testimonies confirm what Magnin and Takahashi (2021, p. 745) have noted about research evaluation when they state that "it not only evaluates the final result of a research paper, but also directs the way of research and publication, the form of writing, the time of submission, in other words, it directs specific practices. This is a control exercised not only over the teacher-researcher, but also over his or her work.
With regard to the creation of the IEC, the committee responsible for planning and carrying out the evaluation of higher education in the universities, Professor Joaquim, who was part of the first committee formed, highlighted the challenges encountered:
Well, in some ways, the IEC is rewarding, but it also has many problems. [...] So when I was on the committee here, it was a time when this part of the IEC was just starting, so we had numerous glitches. [...] As I said, at that time we still had no basis to see what kind of work we were going to do [...] [The lack of experience led us to] seek information from institutions like the Universidade Federal de Juiz de Fora itself, which already had this IEC format so that we would have some experience to get our processes going. [...] So the issue of evaluation, as you know, is very complex, and especially peer evaluation. So we had to gain some experience (Prof. Joaquim).
However, the professor sees an evolution in the work of the IEC in conducting evaluation processes:
But not today. Today I see that it is well founded and that it has, how can I put it... you have plenty of indicators that you can use to do these evaluations. [Today I see that the IEC is more involved in these evaluations (Prof. Joaquim).
For the professor, there are limits to the IEC's work, especially about its relationship with the institution's management teams:
Now, putting into practice what the IEC has developed the results that have been obtained? [...] The IEC has done its job and so on, so that's done, but to really bring it to the community? To discuss it with the community, to bring to the community what the interests are, what the results are, and where you want to go, what you want to achieve, then I really don't think it was 100% legal, no. [...] I think it was more of a rendering of accounts (Prof. Joaquim).
Bernardes (2018) distinguishes between two perspectives on the IEC's work: one that is regulatory and managerial aimed at providing information and accountability, and the other that is emancipatory and democratic, in, which the academic community builds a reflective and conscious evaluation process. According to the testimonies, the work of the IEC on campus is close to accountability. Therefore, strengthening the emancipatory role of this committee and the entire academic community is once again a challenge to be faced.
In conclusion, the data collected show that the context in which the SINAES policy is practiced at the FI studied is experienced by the teachers mainly through the evaluation of the courses. In the statements, vague information was extracted about institutional assessment, as well as silence in relation to Enade, one of the stages that make up this policy. The IEC, the committee in charge of planning, implementing, and disseminating the information resulting from the internal institutional evaluation process, has a timid performance that needs to be strengthened, also by changing the evaluation culture of the institution. The scope of its proposals is limited, as indicated in the testimony.
Teachers complain about the discrepancy between what they do, and the evaluation criteria adopted by the Sinaes policy, which shows that this policy alone does not make it possible to really know the quality of the higher education offered by the FI. This is because, in addition to not properly evaluating what is being done, the policy induces adjustments from the outside in, which do not always result in effective improvements for the courses.
Final considerations
The aim of this article is to explain how the SINAES policy has been implemented in a non-university institution that is equivalent to a federal university in terms of evaluation, supervision, and regulation: the Federal Institute. To this end, a bibliographical and empirical study has been carried out in order to highlight elements of this practice.
With the creation of the FIs, a new institution became part of the Brazilian educational system. With a vertical structure, offering initial and continuing training, basic and higher education in the same space and with the same number of staff, these institutions are present in all Brazilian states, organized in several campuses. Under the influence of international organizations, as Silva (2015) points out, less expensive, low-cost institutions have been created with a flexible educational offer. However, the author warns that teaching in FIs, which are so diverse, becomes a challenge.
According to Otranto (2012), the compulsory offer of degree courses at the FIs, without any counterpart in terms of an increase in human resources (teachers and administrative technicians), combined with the fact that these institutions had little or no experience in teacher training, was identified at the time of the creation of the FIs as a factor in the precariousness and intensification of teachers' work, which is sometimes echoed in the testimonies of teachers more than a decade after the creation of the institution. The lack of incentives for research in the FI campus in question, and the absence of policies to promote this practice - which is provided for in the careers of teachers in basic, technical and technological education, such as those who work in the FIs - reinforces the idea that these institutions were transformed into higher education units before being consolidated as higher education institutions. This reinforces the perception that it is in the market's interest for these institutions to be consumers rather than producers of technological and academic knowledge, making them a cheaper model than universities.
Due to their legal status as federal universities, the FIs have become invisible institutions in the Sinaes policy. In agreement with Viana (2017), Sinaes has not contributed to an adequate evaluation of the FIs, and the university standard applied to such a diverse reality ends up damaging the identity of these institutions, which is still under construction. This, in turn, reinforces the regulatory role of the policy, harming an evaluative, educational, and formative perspective.
However, the regulatory objective of the policy is not a problem in itself, since it is provided for in the law that created Sinaes. The impasse lies in the emptying of the evaluation of its formative aspect, which should problematize the meanings of quality in higher education and encourage improvements in this level of education in Brazil. The limited adoption of the regulatory focus, as explained by Ma and Teixeira (2023, p. 21-22), "instead of acting as an inducer of quality in the formative sense in which Sinaes was created, evaluation would end up encouraging institutions to settle for the minimum levels of quality considered satisfactory".
Finally, the discussions presented indicate the need for further reflection on the provision of higher education in the FIs, as well as the limits and possibilities of adopting a single system to evaluate, regulate and supervise a set of HEIs and courses as diverse as those found in the country, even if this is a recommendation of international organizations interpreted and applied in the Brazilian context.
References
BAL, S. J. Educational reform: a critical and post-structural approach. Buckingham: Open University Press, 1994.
BARDIN, L. Análise de conteúdo. Tradução de Luís Antero Reto, Augusto Pinheiro. São Paulo: Edições 70, 2016.
BARREYRO, G. B.; ROTHEN, J. C. “SINAES” contraditórios: considerações sobre a elaboração e implantação do sistema nacional de avaliação da Educação Superior. Educação & Sociedade, v. 27, n. 96 - especial, p. 955-977, out. 2006.
BARREYRO, G. B.; ROTHEN, J. C. Percursos da avaliação da Educação Superior nos governos Lula. Educação & Pesquisa, v. 40, n. 1, p. 61-76. jan./mar. 2014. DOI: https://doi.org/10.1590/S1517-97022014000100005
BERNARDES, J. dos S. A comissão própria de avaliação: contribuições para o planejamento e para a gestão institucional. In: ROTHEN, José Carlos; SANTANA, Andréia da Cunha Malheiros (org.). Avaliação da Educação: referências para uma primeira conversa. São Carlos: EdUFSCar, 2018.
BORBOREMA, C. D. L. de. Discursos na/da Educação Profissional e Tecnológica: a criação dos Institutos Federais de Educação Ciência e Tecnologia. 2013. 193 f. Tese (Doutorado em Educação) ‒ Faculdade de Educação, Universidade do Estado do Rio de Janeiro, 2013.
BOWE, R.; BALL, S. J.; GOLD, A. Reforming education & changing schools: case studies in policy sociology. London: Routledge, 1992.
BRASIL. Presidência da República. Decreto n.º 5.773, de 9 de maio de 2006. Dispõe sobre o exercício das funções de regulação, supervisão e avaliação de instituições de Educação Superior e curso. Diário Oficial da União, Brasília, DF, 10 maio 2006.
BRASIL. Presidência da República. Lei n.º 9394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da Educação Nacional. Brasil, Brasília, DF, 20 de dezembro de 1996.
BRASIL. Presidência da República. Lei nº 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avaliação da Educação Superior - SINAES e dá outras providências. Diário Oficial da União, Brasília, DF, 15 abr. 2004.
BRASIL. Presidência da República. Lei n.º 11.892, de 29 de dezembro de 2008. Institui a Rede Federal de Educação Profissional, Científica e Tecnológica, cria os Institutos Federais de Educação, Ciência e Tecnologia, e dá outras providências. Diário Oficial da União, Brasília, DF, 30 dez. 2008.
DAROS JÚNIOR, A. O alcance das orientações da Unesco na elaboração do Sistema Nacional da Avaliação da Educação Superior (Sinaes). 142 f. Dissertação (Mestrado em Educação) – Universidade Estadual do Oeste do Paraná, Foz do Iguaçu, 2014.
DIAS SOBRINHO, J. D. de. Qualidade, avaliação: do Sinaes a Índices. Avaliação, v. 13, n. 3, p. 817-825, nov. 2008.
FERREIRA, M.S. Influência de órgãos internacionais na Educação Profissional brasileira. Revista Brasileira da Educação Profissional e Tecnológica, v. 2, n. 13, p. 73-80, 2017.
GOMES, S. dos S.; MELO, S. D. G.; GARRIDO, F. A. Z. Educação Superior na América Latina em tempos de crise. Arquivos Analíticos de Políticas Educativas, v. 30, n. 31, mar. 2022. DOI: https://doi.org/10.14507/epaa.30.7265
MA, J. F.; TEIXEIRA, U. T. Redesenhando o Sinaes: um convite a aperfeiçoamentos na política de avaliação da Educação Superior brasileira. Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, 2023. DOI: https://doi.org/10.24109/1414-0640.td.2023.50
MACEDO, M. N.; ARAÚJO, M. A. D. de. A influência dos organismos internacionais na avaliação da Educação Superior brasileira. Revista Brasileira de Educação, v. 27, e27009, 2022. DOI: https://doi.org/10.1590/s1413-24782022270091
MAGALHÃES, N. M. E.; RODRIGUES, C. M. C. Avaliação e participação na Universidade Federal do Rio Grande do Sul (UFRGS) antes e depois do SINAES: o papel dos Núcleos de Avaliação das Unidades (NAUs). Avaliação, Campinas; Sorocaba, v. 26, n. 01, p. 45-67, mar. 2021. DOI: http://dx.doi.org/10.1590/S1414-40772021000100004
MAGNIN, L. S. de L. T.; TAKAHASHI, A. A política de avaliação da produtividade acadêmica brasileira sob a ótica dos pesquisadores: uma meta-síntese. Avaliação, Campinas; Sorocaba, v. 26, n. 3, p. 742-758, nov. 2021. DOI: http://dx.doi.org/10.1590/S1414-40772021000300006
MAINARDES, J. Abordagem do ciclo de políticas: uma contribuição para análise de políticas educacionais. Educação & Sociedade, v. 27, n. 94, p. 47-69, jan./abr. 2006. DOI: https://doi.org/10.1590/S0101-73302006000100003.
MAINARDES, J.; MARCONDES, M. I. Entrevista com Steven J. Ball: um diálogo sobre justiça social, pesquisa e política educacional. Educação & Sociedade, v. 30, n. 106, p. 303-318, jan./abr. 2009. DOI: https://doi.org/10.1590/S0101-73302009000100015
MAUÉS, O. A política de avaliação da Educação Superior e os desafios da implementação do SINAES. In: REUNIÃO NACIONAL DA ANPEd, GT n.º 11 - Política de Educação Superior, 30., 2007, Caxambu, Anais […]. Caxambu, 2007.
MORAES, M. A. C. de; RAMOS, J. C. S.; OKUYAMA, F. Y.; NUNES, J. G.; PIO, G. da S. O Sinaes nos Institutos Federais: adequação e pertinência no âmbito da avaliação institucional. Revista Brasileira da Educação Profissional e Tecnológica, v.1, n. 6, p. 30-39, 2013. DOI: https://doi.org/10.15628/rbept.2013.3563
OTRANTO, C.R. Criação e implantação dos Institutos Federais de Educação, Ciência e Tecnologia – IFETs. RETTA – Revista de Educação Técnica e Tecnológica em Ciências Agrícolas, v. 1, n. 1, p. 89-108, 2010.
OTRANTO, C. R. Reforma da Educação Profissional no Brasil: marcos regulatórios e desafios. Revista Educação em Questão, Natal, v. 42, n. 28, p. 199-226, jan./abr. 2012.
OTRANTO, C. R. A reforma da Educação Profissional e seus reflexos na Educação Superior. Revista Temas em Educação, v. 22, n. 2, p. 122-135, jan./dez. 2013.
SILVA, P. F. da. A expansão da Educação Superior e o trabalho docente no Instituto Federal do Norte de Minas Gerais. 223 f. 2015. Dissertação (Mestrado em Educação) – Faculdade de Educação, Universidade Federal de Minas Gerais, Belo Horizonte, 2015.
SILVA, S. L. P.; BIFANO, A. C. S. Institutos Federais de Educação, Ciência e Tecnologia e as políticas educacionais orientadas por organismos internacionais. Revista de Educação, Ciência e Cultura, v. 25, n. 3, p. 113-128, nov. 2020. DOI: http://dx.doi.org/10.18316/recc.v25i3.6412
SOUSA, S. M. Z. L. Avaliação e carreira do magistério: premiar o mérito? Revista Retratos da Escola, v. 2, n. 23, p. 81-93, jan./dez. 2008. DOI: https://doi.org/10.22420/rde.v2i2/3.126
VERHINE, R. E. Avaliação e regulação da Educação Superior: uma análise a partir dos primeiros dez anos do Sinaes. Avaliação, v. 20, n. 3, p. 603-619, nov. 2015. DOI: https://doi.org/10.1590/S1414-40772015000300003
VIANA, M. de N. Repercussões do Sinaes no Instituto Federal de Educação, Ciência e Tecnologia do Ceará. 219 f. 2017. Tese (Doutorado em Educação) – Programa de Pós-Graduação em Educação, Faculdade de Filosofia e Ciências da Universidade Estadual Júlio Mesquita Filho, Marília, 2017.
ZATTI, A. M. Avaliação do desempenho docente nas redes públicas estaduais de ensino do Brasil e sua relação com a remuneração. 210 f. 2017. Dissertação (Mestrado em Educação) – Universidade Federal de São Paulo, Escola de Filosofia, Letras e Ciências Humanas, Guarulhos, 2017.
Submission: 09/17/2024.
Approval: 01/03/2025.
[1] Funding: Article published with funding from the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES/Brasil.
[2] Instituto Federal de Minas Gerais, Ouro Preto: fabiula.pires@ifmg.edu.br.
[3] Universidade Federal de Minas Gerais, Belo Horizonte: suzanasgomes@fae.ufmg.br.