# GSoC 2024 - Enhanced Search Functionality for OpenELIS
**URL:** https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731
**Category:** GSOC
**Tags:** gsoc2024
**Created:** [May 7, 2024, 7:41am UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731 "2024-05-07T07:41:23Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [May 7, 2024, 7:41am UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/1 "2024-05-07T07:41:23Z")
Hi everyone, I am excited to start this thread where I will be sharing regular weekly updates on my progress, challenges, and discoveries as I work on my GSoC project.
**Project Name** - Enhanced Search Functionality for OpenELIS
**Project Summary** - This project aims to enhance the search capabilities within the OpenELIS system, particularly the Search Patient module, providing users with a more efficient way of retrieving patient information. We aim to integrate a Java search engine library Hibernate Search with an Apache Lucene backend, to index key columns such as patient ID, patient name and other relevant data, providing users a faster way to search results. This will also include the integration of fuzzy search capabilities which will allow for approximate matching based on similarity, wildcard search which will allow for specified pattern matching and range search which will allow for searching for values within a specified range.
**Mentor** - @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [May 20, 2024, 8:24am UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/2 "2024-05-20T08:24:09Z")
## Community Bonding Period (May 1 - 26)
I had an insightful call with my mentor @Moses_Mutesasira to discuss and refine our project plan, particularly the low-level design of the Search Patient module. Here is a brief summary of the existing and proposed solutions that we discussed -
### Existing solution
The current implementation of the patient search functionality is structured around a controller method _ **getPatientResults** _, a service layer using the interface _ **SearchResultsService** _ and its implementation _ **SearchResultsServiceImpl** _, and a Data Access Object (DAO) using the interface _ **SearchResultsDAO** _ and its implementation _ **SearchResultsDAOImp** _.
### Proposed Solution
- We will create a new implementation of the _ **SearchResultsService** _ interface in the service layer, called _ **LuceneSearchResultsService** _ and a new implementation of the _ **SearchResultsDAO** _ interface, called _ **LuceneSearchResultsDAO** _.
- We will modify the existing service layer implementation _ **SearchResultsServiceImpl** _ to _ **DatabaseSearchResultsService** _ and the existing DAO implementation _ **SearchResultsDAOImp** _ to _ **DatabaseSearchResultsDAO** _.
The coding period officially starts on May 27 and before that, I will focusing on learning more about Apache Lucene search queries and finalising the project plan with my mentor.
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [June 4, 2024, 4:36pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/3 "2024-06-04T16:36:21Z")
# Coding Period - Week 1 (May 27 - June 2)
This week marked the beginning of the GSoC coding period, and I am excited to share the updates of the progress made during this week.
## Focus Areas
- Adding the required dependencies
- Configuring end-to-end setup for Hibernate Search and Lucene index storage
## Accomplishments
-
### Issues
- [GSoC 2024 - Enhanced Search Functionality for OpenELIS](https://github.com/I-TECH-UW/OpenELIS-Global-2/issues/1049)
This issue serves as the main tracker for the project, where all related pull requests will be linked.
-
### Pull Requests
- [Add Hibernate Search dependencies to pom.xml](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1050) **(Merged)**
To configure Hibernate Search, I first added the necessary dependencies to the project’s _pom.xml_ file.
- [Add configuration for Lucene index storage](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1063) **(Merged)**
After adding the dependencies, we needed to specify where Lucene should store the search indexes. This was done through configuration properties in the _persistence.xml_ file.
- [Implement persistent Lucene indexing with Docker volumes](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1064) **(Under review)**
## Challenges
- I faced a blocker when I was experimenting with initialising the Lucene index for _Patient_ entity.
- The error message I encountered was - `Error: HSEARCH600154: Unable to start index: HSEARCH600015: Unable to initialize index directory: /var/lib/lucene_index Context: index 'Patient'`
- After some troubleshooting, I concluded the reason for this error was the user running the application _openelisglobal-webapp_ - **tomcat\_admin** lacked permissions to create the directory.
- I solved this issue by making changes in the _Dockerfile_. I created the necessary directory structure (`/var/lib/lucene_index`) and setting the required ownership and permissions.
## Next steps
- Getting the remaining PR merged by reviewing feedback and making changes if required
- Mapping entities to indexes
- Adding new implementations for the service layer and the DAO
- Modifying the existing implementations
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [June 10, 2024, 10:43am UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/4 "2024-06-10T10:43:26Z")
# Coding Period - Week 2 (June 3 - June 9)
## Focus Areas
- Finishing configuration of setup for Hibernate Search and Lucene index storage
- Mapping entities to indexes
## Accomplishments
-
### Discussions with mentor
This week, I had a productive discussion with my mentor @Moses_Mutesasira during the weekly developer meeting. We discussed on the following topics -
- The fields which should be indexed
- Indexing already existing data if the indexes become out of sync
- Offering choice of Database search and Lucene-based search to the admin user
-
### Pull Requests
- [Implement persistent Lucene indexing with Docker volumes](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1064) **(Merged)**
## Next Steps
- Reviewing and getting feedback from the mentor for the changes addressing the mapping of entities to indexes
- Adding new implementations for the service layer and the DAO
- Modifying the existing implementations
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [June 17, 2024, 5:05pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/5 "2024-06-17T17:05:32Z")
# Coding Period - Week 3 (June 10 - June 16)
## Focus Areas
- Mapping entities to indexes
## Accomplishments
-
### Pull Requests
- [Add bidirectional association for patients in person mapping](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1083) **(Under review)**
- [Add mapping for indexing Patient and Person entities](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1095) **(Under review)**
## Challenges
- I faced a couple of blockers while using the `@IndexedEmbedded` annotation.
- First, I encountered a Hibernate Search exception - `org.hibernate.search.util.common.SearchException: HSEARCH700020: Unable to find the inverse side of the association on type 'org.openelisglobal.patient.valueholder.Patient' at path '.person'.`
- After this change, I encountered a warning message - `Warn: HSEARCH700122: An unexpected failure occurred while configuring resolution of association inverse side for reindexing. This may lead to incomplete reindexing and thus out-of-sync indexes.`
## Next Steps
- Getting the PRs merged by reviewing feedback and making changes if required
- Adding new implementations for the service layer and the DAO and modifying the existing implementations
- Designing and implementing Apache Lucene search queries
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [June 25, 2024, 1:05pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/6 "2024-06-25T13:05:17Z")
# Coding Period - Week 4 (June 17 - June 23)
## Focus Areas
- Finishing mapping of entities to indexes
- Adding the required tests related to the changes
## Accomplishments
-
### Pull Requests
- [Add bidirectional association for patients in person mapping](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1083) **(Merged)**
- [Add mapping for indexing Patient and Person entities](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1095) **(Merged)**
## Next steps
- Adding new implementations for the service layer and the DAO and modifying the existing implementations
- Designing and implementing Apache Lucene search queries
- Adding support for manually re-indexing the already persisted data using Hibernate Search `MassIndexer`
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [July 3, 2024, 4:50pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/7 "2024-07-03T16:50:11Z")
# Coding Period - Week 5 (June 24 - June 30)
## Focus Areas
- Adding new implementations for the service layer and the DAO and modifying the existing implementations
## Accomplishments
-
### Pull Requests
- [Add new implementations for the search results service and the DAO and modify the existing implementations](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1150) **(Under Review)**
- [Change index directory type to local heap in testing configuration](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1144) **(Merged)**
## Next steps
- Designing and implementing Apache Lucene search queries
- Adding support for manually re-indexing the already persisted data using Hibernate Search `MassIndexer`
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [July 10, 2024, 12:38pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/8 "2024-07-10T12:38:43Z")
# Coding Period - Week 6 (July 1 - July 7)
## Focus Areas
- Designing and implementing Apache Lucene search queries
## Progress
- I am currently working on implementing Lucene queries in `getSearchResults` and `getSearchResultsExact` methods of `LuceneSearchResultsDAOImpl` class. I will soon raise a draft PR for the changes.
- This change also includes tests for the `LuceneSearchResultsServiceImpl` which will replace the existing placeholder tests.
## Next steps
- Reviewing feedback from the mentor and getting the PR finalised
- Adding support for manually re-indexing the already persisted data using Hibernate Search `MassIndexer`
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [July 21, 2024, 5:32am UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/9 "2024-07-21T05:32:13Z")
# Coding Period - Week 7 and 8 (July 8 - July 21)
## Focus Areas
- Finishing implementation of Apache Lucene search queries
## Accomplishments
-
### **Pull Requests**
- [Implement Lucene queries in getSearchResults and getSearchResultsExact methods of LuceneSearchResultsDAOImpl](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1180) **(Merged)**
- I will soon create a PR to make `LuceneSearchResultsServiceImpl` the primary implementation for `SearchResultsService` interface.
## Next steps
- Adding support for manually re-indexing the already persisted data using Hibernate Search `MassIndexer`
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [August 7, 2024, 6:15pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/10 "2024-08-07T18:15:25Z")
# Coding Period - Week 9 and 10 (July 22 - August 4)
## Focus Areas
- Adding support for manually re-indexing the already persisted data using Hibernate Search `MassIndexer`
## Accomplishments
-
### **Pull Requests**
- [Make LuceneSearchResultsServiceImpl the primary implementation for SearchResultsService interface](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1218) **(Under Review)**
- [Add support for manual re-indexing using MassIndexer](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1220) **(Draft)**
CC: @Moses_Mutesasira
---
### Author:  [@rahul6603](https://talk.openelis-global.org/u/rahul6603)
#### Post date: [August 23, 2024, 8:12pm UTC](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis/731/11 "2024-08-23T20:12:06Z")
# Coding Period - Week 11 and 12 (August 5 - August 18)
## Focus Areas
- Finishing adding support for manual re-indexing
- Adding Documentation
## Accomplishments
-
### Pull Requests
- [Add support for manual re-indexing using MassIndexer](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1220) **(Merged)**
- [Add test for MassIndexerService](https://github.com/I-TECH-UW/OpenELIS-Global-2/pull/1240) **(Draft)**
-
### Documentation
- [Project Report](https://gist.github.com/rahul6603/1359765bedb0fa1957700536516e7ca6)
- [Final Presentation](https://talk.openelis-global.org/t/gsoc-2024-enhanced-search-functionality-for-openelis-final-presentation/843)
CC: @Moses_Mutesasira