Maricopa continues to work with the vendor that is developing the Grade Export building block. Testing is going well and every indication is that it will be ready to be installed in the Blackboard production environment sometime in July.
The performance impact of the Grade Export Building Block is being rigorously tested using the Empirix tool to simulate students on the Test system while the export is being executed.
Friday, June 29, 2007
Monday, June 11, 2007
Initial Fall 2007 Course Creation Completed
Over the weekend (6/9 & 6/10) the initial Fall 2007 course creation completed. The initial phase of course creation was done while Maricopa's Blackboard was experiencing issues with the file server (see post below for details). This weekend run completes the creation of all courses that are currently flagged, bringing the number of Fall courses to just over 5700.
Beginning 6/11/2007 the Fall 2007 course creation will now be included in the nightly inteface run, any new courses for Fall 2007 that are flagged during the day will be created that night.
Beginning 6/11/2007 the Fall 2007 course creation will now be included in the nightly inteface run, any new courses for Fall 2007 that are flagged during the day will be created that night.
Friday, June 1, 2007
Bb Performance Problems after Upgrade to 7.1
The Bb upgrade from 7.0 to 7.1 involved a lot more than simply upgrading the Bb application. Many infrastructure changes were also done during the upgrade (5/22/07 - 5/23/07). To better handle the number of small files in Blackboard the infrasture team recommended that the back end file server be changed from the current configuration to a new file server using the Veritas file system. Testing showed that throughput to the Veritas file system was much better than to the old file system. Backups from the Veritas file system fininshed in about half of the time as the old file system.
However, after upgrading to the Veritas file system Bb performance slowed to a crawl as the Bb application seemed to wait for files from the file system. After much research, and with key technical help from Jeremy at Bb, it was found that while regular file IO is faster on the Veritas file system, when Java access files for Bb that file system was much slower responding to file attritibute requests (does the file exist, does Bb have access to the file, etc).
Testing confirmed that when a Bb 7.1 web/app server was pointed at an "old" file server with current data that Bb performance problems went away. To resolve the issue a plan was developed the move the current 7.1 data to a new LUN on the EVA that would be accessed through the "old" file system (but with a larger LUN). This plan was executed the night of 5/30 and into 5/31. The morning of 5/31 when Bb was pointed at the old file server with the current data in the larger volumne, the probem was resolved.
This problem may have been avoided if the test environments for Bb more closely mirrored the production environment at the infrastructure level. Due to the limitations of the "old" file system this issue still must be addressed. The old file system takes too long to backup and there is the danger that there will simply be too many files for the file system to support (especially in the "courses" folder). It is likely that the file server configuration will be changed at the end of the Summer I semester, the night of June 28th.
However, after upgrading to the Veritas file system Bb performance slowed to a crawl as the Bb application seemed to wait for files from the file system. After much research, and with key technical help from Jeremy at Bb, it was found that while regular file IO is faster on the Veritas file system, when Java access files for Bb that file system was much slower responding to file attritibute requests (does the file exist, does Bb have access to the file, etc).
Testing confirmed that when a Bb 7.1 web/app server was pointed at an "old" file server with current data that Bb performance problems went away. To resolve the issue a plan was developed the move the current 7.1 data to a new LUN on the EVA that would be accessed through the "old" file system (but with a larger LUN). This plan was executed the night of 5/30 and into 5/31. The morning of 5/31 when Bb was pointed at the old file server with the current data in the larger volumne, the probem was resolved.
This problem may have been avoided if the test environments for Bb more closely mirrored the production environment at the infrastructure level. Due to the limitations of the "old" file system this issue still must be addressed. The old file system takes too long to backup and there is the danger that there will simply be too many files for the file system to support (especially in the "courses" folder). It is likely that the file server configuration will be changed at the end of the Summer I semester, the night of June 28th.
Subscribe to:
Posts (Atom)